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Chapter  1 


INTRODUCTION 

With  the  advent  of  efficient  and  large-scale  mathematical  program¬ 
ming  techniques,  computationally  feasible  methods  are  available  for  op¬ 
timal  control  problems.  The  purpose  of  this  paper  is  to  present  an 
algorithm  for  solving  continuous-time  optimal  control  problems  with 
linear  dynamics  and  various  loss  criteria.  Due  to  the  mathematical 
programming  techniques  used  in  the  algorithm,  it  is  well  suited  for 
large-scale  control  problems,  i.e.,  control  problems  with  large  numbers 
of  state  variables  and  time-varying  control  inputs.  This  work  consists 
of  two  main  results  that  are  combined  to  develop  the  algorithm. 

In  Chapter  II,  we  describe  the  types  of  control  problems  considered, 
including  basic  definitions  and  notations  for  these  problems.  The  basic 
results  in  control  theory  and  certain  necessary  conditions  for  optimal 
control,  as  dosr  .•  i  bed  by  Pontryagin  et  al.  Til,  are  also  presented. 

In  Chapter  III,  t  lie  algorithms  and  basic  theorems  for  linear  pro¬ 
gramming  and  the  simplex  method  [21,  quadratic-  programming  and  the 
complementary  pivot  theory  [Til,  and  the  Dantzig-Wol fe  generalized  pro¬ 
gram  | 21  are  presented. 

The  iirst  main  result,  an  algorithm  for  solving  parametric  linear 
and  quadratic  programming  problems,  when  the  objective  function  is  non¬ 
linear  in  the  parameter,  is  presented  in  Chapter  IV.  Also  presented  is 
the  class  of  nonlinear  functions  for  which  this  algorithm  is  valid.  The 
finiteness  of  the  algorithm,  including  avoidance  of  cycling  due  to  de¬ 
generacy,  is  then  proven.  The  characteristics  of  the  optimal  solution 
as  a  function  of  the  parameter  are  also  described. 

The  second  result,  an  extension  of  Dantzig's  T  !'l  formulation  of 
optimal  control  problems  as  generalized  programs,  is  presented  in  Chap¬ 
ter  V.  It  is  shown  t  ha  t  any  optimal  control  problem  with  the  following 
characteristics  may  be  formulated  as  a  generalized  program:  (1)  the 
system  must  initiate  from  some  point  in  a  specified  region  of  the  slate 
space;  (2)  the  state  at  t  lie  fixed  terminal  time  can  be  chosen  from  an¬ 
other  convex  region  in  t  lie  state  space  (fixed  initial  and  final  points 


are  included  in  these  definitions);  (3)  the  state  of  the  system  is  con¬ 
trolled  by  linear  differential  equations;  (4)  the  admissible  control 
region  is  a  convex  polyhedral  set  (for  each  point  in  time)  in  the  con¬ 
trol  space;  (5)  the  loss  criteria  is  a  linear  functional  in  the  state 
and  control  and/or  a  quadratic  functional  in  control  and/or  the  absolute 
value  of  the  control  inputs  (minimum  fuel),  or  the  minimum  time.  It  is 
further  shown  that  these  continuous-time  optimal  control  problems  have 
an  equivalent  generalized  programming  formulation  in  which  the  master 
problem  linear  program  o.  or  throe  plus  the  di......  f  11^ 

state  space.  The  subproblem  to  the  master  program  is  a  parametric  pro¬ 
gramming  problem  of  the  control  space  dimension  and  is  solvable  by  the 
methods  presented  in  Chapter  IV.  This  subproblem  yields  an  extreme  ad- 
missable  control  that,  when  used  with  previously  found  extreme  admis- 
sable  controls,  gives  a  solution  t ha t  is  closer  to  a  feasible  or  an 
optimal  one. 

The  algorithm  and  its  variants  >re  presented  in  the  second  part  of 
Chapter  V.  A  flow  chart  of  the  algorithm  is  given,  along  with  a  descrip¬ 
tion  of  eacli  execution.  Also  included  is  an  initialing  phase  that  ter¬ 
minates  in  a  feasible  solution  of  the  control  problem.  On  completion  of 
the  initiating  phase,  the  algorithm  maintains  a  feasible  control  while 
obtaining  new  controls;  those  now  controls  yield  better  objective  values 
without  disturbing  the  feasibility.  1'pper  and  lower  bounds  on  the  op¬ 
timal  objective  value  arc  provided  at  each  stage  of  the  algorithm. 

In  Chapter  VI,  the  characteristics  of  the  optimal  controls,  without 
any  additional  assumptions  on  the  system  or  on  the  uniqueness  of  t  lie 
solution,  are  presented.  Also  included  are  the  relationships  between 
the  necessary  conditions  of  Pontryagin  and  the  generalized  programming 
results.  Between  these  optimization  conditions,  a  link  exists  in  the 
dual  variables  of  the  generalized  program  and  the  ad.ioint  variables 
associated  with  the  optimal  control  problem. 

To  clarify  the  algorithm  and  indicate  its  computational  feasibility, 
a  minimum  fuel  problem  and  a  minimum  time  problem  arc-  solved  in  detail 
in  Chapter  VII.  The  convergence  properties  and  solution  procedures  are 
illustrated  with  data  obtained  from  computer  runs. 


Chapter  II 


OPTIMAL  CONTROL 

This  section  defines  an  optimal  control  problem  and  Pontryagin's 
necessary  conditions  for  optimality.  The  emphasis  is  on  those  linear 
systems  for  which  generalised  programming  equivalents  can  be  formulated. 

A.  Definition  of  Dynamic  Control  Systems 

The  basic  control  problem  can  be  described  by  the  differential 
equations: 


dx . 

*i  =  “dT  =  Vxi . V  ui . V  t} 


i  —  1 ,  2,  ....  n  , 


(2.1) 


where 


Pl(,)l 


x  ( t )  = 


x  (O 
n 


(2.  2) 


is  the  vector  of  state  variables  or  phase  coordinates  which  describe  the 
trajectory  of  the  system  in  Euclidean  space  through  time.  The  control 
function  is  the  vector  of  control  inputs 


rv’i 


u  (  l  )  = 


u  (t) 
m 


(2.  3) 


which  influence  the  state  through  the  differential  equations.  The  sys¬ 
tem  at  some  initial  time,  t  ,  satisfies  t  lie  initial  conditions, 

o 


(2.  4) 


x( t  )  t  S  C  EU  . 
o  o 


U 

•I 

Thus  the  system  may  have  an  initial  point  x(tQ)  at  any  one  of  the  points 

in  the  set,  !S  .  At  a  terminal  time,  T,  the  system  is  required  to  lie 
o 

in  some  region,  i.e,, 


x(T)  e  STQEn  .  (2.5) 

The  time  T  may  be  free  or  fixed,  and  the  sets,  and  S^,  may  be 

fixed  points. 

B.  Admissible  and  Feasible  Controls  and  Reachable  Sets 

The  vector  control  function,  u(t)  must  be  specified  at  every  t 
and  is  required  to  lie  in  an  admissible  control  region,  U^,  where 

u(t)  e  Ut  c  Em  ,  vt  .  (2.6) 

Definition  2.  1.  An  admissible  control  is  any  vector  function,  u(t), 
for  which 


u(t)  t  U  C  Em  ,  i/t  t  [0,T]  , 


wliere  [0,Tl  denotes  the  time  interval  (i|0  <  t  <  T)  . 


The  objective  of  the  control  problem  is  to  find  an  admissible  con¬ 
trol  function  that  transfers  the  state  from  some  point  at  t  to  an- 

o 

other  point  at  T,  while  minimizing 


J 


f 

o 


x  ,  u  ,  . . . ,  u  ,  t)  dt  . 
n  1  m 


(2.  7) 
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It  is  convenient  at  this  point  to  define  another 


variable  , 


X°(t;  =  X  f°(Xl’  V  V  •••’  Um’  °  dt  > 


(2.8) 


and  to  let 


x(t)  = 


x  (t) 
o 


Lx(t)  J 


(2.9) 


Thus, 


Xo  =  fo(x>  u*  t)  .  VV  =  0  . 


and  J  =  x  (T) 
o 


(2.10) 


JMAnitionj^.  The  na«^  3£t|  dcnoted  by  consists  of  a  ^ 


of  terminal  x(T)  of  admissible  solutions 


without  the  condition  x(T)  t  s 

T* 


to  the  control  problem, 


R  =  {x  t  £  |x  =  x(T)  , 


where  x(T)  is  a  solution  to  (2.1)  at  t  =  T  with 


s0  •  i'(t)  t  U  ,  Vt} 

Note  that  for  the  fixed  final  time,  T,  if 


s  n  r  = 
T  T 


there  is  no  admissible  control  to  transfer 


point  in  S  to 
o 


the  system  from  an  initial 


a  point  in  S 


if 

Is 


— - - - --  ---■ 


F’diimiKiiiiiirri  I  asa 


Definition  2.3.  A  control  function,  u(t),  defined  for  t  t  [t  , T]  is 
— — — — o 

a  feasible  control  for  the  optimal  control  problem  if  it  is  an  ad¬ 
missible  control  and  transfers  the  system  from  some  state  x(t  )  e  S 

o  o 

to  a  state  x(T)  t  while  x(t)  satisfies  (2.1).  Note  that  a 
feasible  control  exists  iff 


sT  n  rt  *  0  . 


In  the  optimal  control  problem,  we  are  searching  for  a  control 
function,  among  all  feasible  controls,  that  results  in  a  minimal  value 
of  J. 


Assumption  2.  1.  Y/e  will  now  restrict  our  attention  to  functions  f  , 

for  i  =  0,  ...,  n,  which  are  autonomous,  i.e.,  they  do  not  depend 
explicitly  on  time.  We  will  also  assume  that  the  f.  functions  for 
i  =0,  1 . n,  are  continuous  in  both  x  and  u  and  are  con¬ 

tinuously  differentiable  with  respect  to  x. 


C.  The  Adjoint  System  and  the  Hamiltonian 


For  any  given  u  or  x(o) ,  let  x  =  x(t)  be  determined  by 


x i  =  f  (x,  u)  i  =  0,  1 ,  .  .  .  ,  n  . 


For  this  choise  of  u,  x(o),  and  the  resulting  x(t),  we  define  the 

adjoint  system,  y  ,  y .  T  ,  by 

U  1  n 


dj'.  "  (x,u) 

i  \  k 


=  ir = -  y 


i  =  0,  1 . n  , 


(2. 11) 


where  the  partials  are  evaluated  at  the  above  x(t),  u(t).  The  solution 
to  (2.11)  is  related  to  the  choice  of  control,  u(t). 
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The  Hamiltonian  is  defined  as 


H('p ,x,u)  =  \J/'  f(x,  u) 
where  T '  is  the  transpose  of 


'i 


and 


f  (x,u) 
o 

f1 (x, u) 

f(x,u)  -  I 


f  (x,u) 
n 


Thus  (2.1)  and  (2.11)  become 


x 

i 


1 


(2.  la) 


i  .  = -  (2.  11a) 

1  'x . 


D.  Pontryagin ' s  Conditions  for  Opt i  ma  1  i  t  y 

When  the  initial  and  final  points,  x(t  )  and  x(T),  arc  not 

o 

fixed,  the  regions  S  and  S  are  assumed  to  lie  smooth  manifolds  or 

O  O  I 

convex  sets.  A  necessary  condition  for  optimality  in  this  case  is  that 
the  solution  to  (2.1)  and  (2.11)  satisfy  a  t ransversa 1 i t y  condition. 
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Let  x(t  )  e  S  and  x(T)  e  S  be  given  points  on  the  boundary  of  S 
o  o  T  o 

and  S  ;  and  let  D  and  D  be  tangent  planes  of  S  and  S  at 
T  o  T  o  T 

these  points.  Then  the  solution  satisfying  (2.1)  and  (2.11)  will  also 

satisfy  the  transversality  condition,  if  'J' (  t and  f(T)  are  the 

directions  of  the  supporting  hyperplanes,  D  and  D  .  of  S  and 

o  T  o 

S  at  x(t  )  and  x(T) ,  respectively. 

T  o 

Necessary  Conditions  for  t  <  t  <  T.  Let  u(t)  be  a  feas¬ 
ible  control  with  a  corresponding  trajectory  x(t).  For  u(t)  =  u*(t) 
to  yield  an  optimal  solution  to  the  control  problem,  it  is  necessary  to 
have  a  non-zero  continuous  vector  function  f(t)  corresponding  to  x(t) 
and  u*(t),  (2.1)  and  (2.11),  and  satisfying  the  transversality  condi¬ 

tions  so  that 


(1)  For  t  t  [t  ,T], 
o 


H  [  x  ( t )  , 


u*(t) , 


'1  ( t )  ] 


Sup 

u(t)tU 


H[x( t ) ,  u ( t ) ,  T(t)l 


and 


(2)  f  (T)  <  0 
o  — 


E.  The  Linear  System  and  Control  Constraints 

A  linear  system  is  defined  as  a  dynamic  system  in  which  the 

f  (x  ,  .  .  ,  x  ,  u .  u  )  are  linear  in  x  and  u  for  i  —  1 , 

i  1  n  1  in 

n.  Note  ttiat  f  (x,u)  need  not  be  linear.  This  linear  system  can  be 
o 

described  by  two  matrices,  F  and  G,  as 

x  ( t )  =  F  x  ( t )  +  Gu  (  t )  ,  x  ( t )  ,  En  ,  u(t)  ,  E1"  ,  (2.12) 


where  F  is  an  n  y  n  real  matrix  and  G  is  an  n  y  m  real  matrix. 

The  linear  system  has  a  fundamental  matrix  [dl  e^ 1  ^  that  has 
the  property  of  transforming  x(t)  by: 
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u(t)  0  for  T  c  [t0,ti].  This  fundamental  matrix  arises  from 
the  solution  of  the  differential  equations  in  (2.12)  when  u(t)  =  0. 

The  solution  for  any  function  u(?)  is 


F(W 

xdj)  =  e  °  x(t 


“V  *  f 

to 


■tj  F(t  -O 
e 


Gu(t)  dt  .  (2.13) 


«.en  u(„  t  ut  for  all  ,  „„d  x(,0)  ,  S(>,  tho  r  1K„ t -„a n(l  Bide 

Of  (2.13)  determines  a  point  in  the  reachable 


V  Hence  we  can  slate,  for  linear  s 


e  set  of  U  ,  S  ,  and  time 

t  o 


systems, 


-  i  .  Kn| 


R.  =  /x  t  K  lx  =  x(t  )  , 


F(  t  -t . ) 


f  F(t  -;) 


v/.  \  i  o'  r1 1 

x(t1)  =  c  x(t0)  +  /  1  e  1 

-A 


Gu ( ; )  d i 


u(i)  v  r 


,  -  1  ^  fVV  -  x(t0)  ,  So 


Throughout  this  paper,  we  will  consider  problems  where  l<  , 
UCE  ,  i.o. ,  the  admissible  control  set  is  constant  over  time!  We 
also  assume  that  U  is  a  bounded  convex  polyhedral  set,  i.o.,  it  is 
bounded  by  hyperplanes  in  m-dimensionn  1  space.  Note  that  any  convex 
polyhedral  set  can  be  expressed  by 


f  -  u  E  !  All  <  b) 


for  some  real  iixed  matrix  A  of  dimension 
vector  b  of  dimension  q. 


9  *  m  and  for  some  real 
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In  the  following,  we  will  permit  the  initial  and  terminal  sets,  S 

and  S  c  E*1,  to  be  convex  sets. 

Note  that  f.(x,u)  =  f'x  +  g'u,  wliere  f  is  an  n-dimensional 

1  i  i  l  ^ 

vector  and  g  is  an  m-dimensional  vector.  f.  and  g  are  the  i 

i  i  i 

rows  of  the  F  and  G  matrices,  respectively. 

F.  Loss  Functionals 

In  this  section,  we  will  describe  the  different  classes  of  loss 
functionals.  These  loss  functionals,  when  combined  with  linear  systems 
and  the  above  restrictions,  can  be  solved  by  mathematical  programming 
techniques  that  are  developed  and  discussed  in  the  next  two  chapters. 


Case  1.  Linear  Loss  Functionals. 

We  define  the  linear  loss  case  as  one  that  includes  all  loss  func¬ 
tionals  of  the  form 


f  (x,u)  =  f'x  +  g'u  , 
o  o  o 


wliere  f  and  g  are  any  real  n 
o  o 

tively.  Thus,  we  can  define  linear 
as  complet'ly  linear  systems. 


and  m  component  vectors,  respec- 
systems  with  linear  loss  functionals 


Case  2.  Minimum  Fuel  Problems. 

A  certain  well-known  minimum  fuel  problem  is  characterized  by  loss 
functionals  of  the  form 


f(u) 


Case  3.  Quadratic  Loss  in  Control. 

We  consider  a  function  a  quadratic  only  in  the  control  vector, 

i.  e.  , 
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f(u)  =  u'Qu  , 


where  Q  is  a  positive  semidefinite  matrix. 

When  a  linear  functional  is  added  to  f(u)  and  modifications  of 
Cases  2  and  3  are  permitted,  the  three  cases  are: 


f  (x, u)  =  f ' x  +  g'u  +  f ( u ) 

o  o  o 


(° 

Case  1 

where  f  ( u)  =  <  II  u 

,  Case  2 

)  ' 

1  u '  Qu 

,  Case  3 

(If  f  (x,u)  =  f(u),  then  f  =0,  and  g  =0.)  Thus  the  control 
o  o  o 

problem  can  be  stated  as 


where 


Minimize  x  (T) 
o 


x 


Fx( t )  +  Gu ( t )  +  f ( u)  V 

o 


(2.1-1) 


and 
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x( t  )  e  S  ,  x  ( t  )  =  0 
o  o  o  o 

x(T)  e  ST  ,  and 


u(t)  t  U  =  (u  t  Em|  Au  <  b)  . 


The  minimum  time  problem  is  also  considered  where  T  is  not  fixed, 

S  and  are  fixed  points,  and  x  (T)  =  T  -  t  . 

o  T  o  o 

For  linear  systems  described  by  matrices  F  and  G  and  a  given 
polyhedron,  u,  Pontryagin  defines  a  "general  position  condition."  This 
condition  is  satisfied  when  the  vectors  Gw,  FGw,  ...,  F11  *Gw  are  lin¬ 
early  independent  in  En  when  w  has  the  direction  of  one  of  the  edges 
of  U.  For  such  systems,  at  each  point  of  time,  t,  the  function 
i(t)'Gu(t)  achieves  its  maximum  at  only  one  vertex  of  U,  except  on  a 
set  of  measure  zero. 

Before  pioceeding  further  with  the  development  of  an  algorithm  to 
solve  these  continuous-time  control  problems,  some  of  the  existing  tech¬ 
niques  used  in  solution  procedures  should  be  mentioned  briefly.  Three 
of  these  techniques  are  mentioned  here. 

Direct  Methods  [5] .  In  these  methods,  admissible  and,  if 
possible,  feasible  controls  are  chosen  to  start.  The  gradient  of  the 
cost  functional  (or,  if  the  starting  control  is  not  feasible,  a 
Lagrangian  form  that  takes  feasibility  into  account),  witli  respect  to 
the  control  function,  is  determined .  Then,  by  using  gradient  or  steepest 
descent  methods,  a  new  control  function  is  chosen  to  improve  the  cost 
functional  (or  Lagrangian)  . 

Indirect  Methods  [6],  Indirect  methods  primarily  seek  solu¬ 
tions  to  the  necessary  conditions  for  optimality.  Some  methods  use 


12 


arbitrary  initial  or  final  conditions  for  the  adjoint  variable.  In  this 

case,  the  differential  equations,  (2.1)  and  (2.11),  are  integrated  to 

find  solutions  for  x(t  );  during  this  procedure,  a  solution  to  the 

o 

necessary  conditions  is  retained,  if  possible.  If  x(t  )  is  not  equal 

o 

to  the  original  (known)  x*(t  ),  the  gradient  of  some  cost  functional, 

o 

based  on  the  distance  from  x(t  )  to  x*(t  ),  is  used  to  determine  a 

o  o 

new  guess  for  the  final  time  adjoint  variable  values. 

Discrete  Approximations  [7,8],  Mathematical  programming  tech¬ 
niques,  e.g.  ,  linear  programming  or  gradient  projection  methods,  are 
usually  applied  to  a  discrete  approximation  of  the  continuous-time  prob¬ 
lem.  In  these  approximations,  the  system  is  considered  at  a  prescribed 

set  of  instants  in  the  interval  [t  ,  T].  Only  at  these  times  are  the 

o 

control  inputs  allowed  to  change.  The  differential  equations  are  then 
approximated  by  difference  equations  for  each  time  considered.  Math¬ 
ematical  programming  techniques  are  then  used  to  solve  the  approximation. 

Each  of  the  three  techniques  mentioned  have  their  disadvantages. 

The  direct  methods'  disadvantage  is  that  a  feasible  control  must  be  pro¬ 
vided  initially.  II  not,  the  convergence  methods  cannot  be  guaranteed  to 
terminate  with  a  feasible  solution.  Also,  the  efficiency  of  convergence 
is  highly  dependent  on  the  initial  guess.  The  indirect  methods  also  have 
a  disadvantage  in  that  they  do  not  provide  a  feasible  solution  until  the 
final  step.  At  times,  the  determination  of  a  feasible  solution  is  the 
major  problem  in  optimal  control.  The  basic  disadvantage  of  discrete 
approximations  stems  from  the  large  number  ol  variables  or  equations 
introduced  by  the  approximation  process. 

The  methods  developed  in  this  work  combine  the  features  of  both  the 
direct  and  indirect  methods  and  use  admissible  controls  to  find  a  fea¬ 
sible  solution.  This  combination  continuously  reduces  the  cost  while  it 
retains  the  feasibility  and  converges  on  the  optimum  values  of  the  ad¬ 
joint  variables.  Thus  the  problem,  at  any  iteration  in  the  optimization 
phase,  has  a  feasible  solution  available,  and  the  present  solution  has  a 
measure  of  closeness  to  the  optimum  solution  [91. 


Chapter  I  1 1 


MATHEMATICAL  PROGRAMMING 

In  this  chapter,  t he  algorithms  available  for  solving  linear  and 
quadratic  programming  are  reviewed,  and  the  theory  of  generalized  pro¬ 
gramming  is  described.  The  choice  of  the  simplex  method  for  linear 
programming  problems  and  the  complementary  pivot  theory  for  quadratic 
programming  problems  is  dictated  by  t  lie  ease  encountered  in  using  the 
parametric  programming  methods  presented  in  Chapter  IV. 

It  should  be  noted  tiiat  any  bounded  convex  polyhedral  set  can  be 
represented  (possibly  after  a  change  ol  variable)  by  the  set 
X  =  (x|Ax  <  b,  x  >  0]  for  some  real  matrix,  A,  and  for  some  real 
vector,  b. 

A.  Linear  Programming 

The  standard  linear  programming  problem  can  be  s'ated  as 

minimize  z  -  c'x 

subject  to  Ax  <  b  and  (3.1) 

x  >  0  , 

n 

where  x  E  ,  c  is  a  specified  n-tl  i  mens  i  ona  1  vector,  b  is  a  speci¬ 
fied  m-d i mens i onal  vector,  and  A  is  a  given  (m  n)  matrix. 

Since  minimizing  c'x  is  equivalent  to  maximizing  (-c')x,  only 
minimization  problems  are  discussed.  Hence,  problem  (3.1)  seeks  the 
minimum  of  a  lineai  (convex  and  concave)  function  over  a  convex  poly¬ 
hedral  constraint  set;  if  the  :atter  is  nonempty,  a  solution  exists 
and  is  known  to  be  at  an  extreme  point  in  the  constraint  set.  Thus  we 
need  only  consider  basic  solutions  to  problem  (3.1),  i.e.,  solutions  in 

which  no  more  than  m  components  ol  the  vector  x  are  positive  and 
whose  column  coefficients  are  linearly  independent  in  rows  where  Ax  <  b 
is  satisfie'  with  equa  i t y. 


The  dual  problem  to  (3.1)  can  be  expressed  by 
minimize  v  =  by 

subject  to  A'y  >  c  (3.2) 

y  >  0  ,  y  t  Em  . 

The  duality  theorem  of  linear  programming  can  be  summarized  in  two 
statements: 

(1)  for  any  feasible  x, y  [satisfying  the  constraints  of  (3.1) 
and  (3.2)1, 


c '  x  >  b' y  ,  and 

(2)  for  the  optimal  x*,  y*  of  (3.1)  and  (3.2), 

c  '  x*  =  b'  y* 

(Ax*  -  b) ' y*  =  0  j 

/  complementary  slackness  conditions  . 

(A'y*  -  c)  '  x*  =  0  ) 

If  the  x  vector  is  augmented  by  m  components  to  include  slock  vari¬ 
ables  and  the  matrix  A  is  augmented  by  I,  the  constraint  inequalities 
are  equivalent  to 


Ax  =  b 

x  "■  0  , 


(3.  3) 


where  A  and  x  are  now  the  augmented  matrix  and  vector,  respectively. 

Since  we  need  only  investigate  the  extreme  points  of  the  constraint 
set,  we  need  only  allow  basic  solutions  corresponding  to  choosing  m 
linearly  independent  columns  of  A, and  the  components  of  the  vector  x 
corresponding  to  the  m  columns  of  A.  The  m  columns  of  the  augmented 
A  form  a  nonsingular  matrix  B,  called  the  basis  matrix.  The  corre¬ 


sponding  components  of  x  are  called  the  basic  variables. 


Hence,  a 


basic  feasible  solution  to  (3.1)  is  one  in  which  the  values  of  the  basic 
variables  are  nonnegative,  and  all  the  other  variables,  called  the  non- 
basic  variables,  are  at  value  zero.  Let  xQ  represent  the  vector  of  the 
basic  variables  corresponding  tc  B.  Then  the  basic  solution  to  the 
linear  equations  in  (3.3)  is 


where  i  is  nonbasic.  This  is  a  basic  feasible  solution,  provided 
XI3  -  °- 

The  Simplex  Method.  The  simplex  method  is  reviewed  in  detail, 
since  a  variation  of  it  is  employed  in  Chapter  IV  for  the  parametric 
programming  procedures.  This  method  is  presented  in  matrix  form.  Here, 
the  linear  program 


minimize  z  =  c'x 

subject  to  Ax  =  b  (3.4) 

x  _  0 

is  observed,  and  the  augmented  system  of  equations 


(3.5) 


is  used. 


Given  any  basis,  B,  let  t  lie  augmented  basis  be  B,  where 


B  = 


1  ~C  B 


0  B 


17 


and  rewrite  (3.5)  as 


where 


— 

—  — 

r—  — 

1 

-c ' 

-c 1 

z 

0 

B 

R 

XB 

= 

0 

B 

R 

_XR_ 

b 

— 

— 

_  _ 

0 


A 


and 


c 


B 


c  = 


cR  is  the  vector  of  the  components  of  c  corresponding  to  t  lie  basic 
variables  x  . 

Since  B  is  nonsingular,  B  is  also  nonsingular; 

_-l 
B 


Multiplying  (3.6)  by  B  "  and  then  rearranging  il,  we  get 


1  en" 


0  B 


-1 


By  setting  the  nonbasic  variables,  x  ,  at  level  zero, 


1  cbb 


0  B 


CB°  b  CB*B 


B~1b  B  1|, 


where  x^  =  B  b. 

If  B  ^ b  is  a  nonnegative  vector,  the  basis  B  is  feasible, 

and  thus  the  current  solution  is  a  basic  feasible  solution. 

Look  at  any  variable  x.  with  a  corresponding  column  A 

J  j 

and  a  cost  coefficient  c  ;  this  variable's  column  in  the  transformed 

J 

system  of  (3. 7)  is 


-c ,+c '  B 
J  B  .j 


If  x.  is  a  basic  variable,  »  \\  is  the  r"'  unit  vector,  ll  A 
.  1  t  h  '  i 

13  the  r  colunui  of  B.  (Note  that,  in  this  case,  c.  would  be  the 

t  h  t  h  i 

r  component  of  c  ,  and  x  would  be  t  lie  r  component  of  x  ) 

II  1  B 

Thus  the  first  component  of  (3.K)  becomes 


-c  ,  t  c'B  A  -  -c  -t  c  -  0 
1  h  j  )  j 


moreover  (3.8)  is  a  unit  vector. 


Proposition  3.1.  I  (  all  c.  -  e.'.B  A  0.  t  lie  current  basis  It  is 
-  1  11  )  - 


opt  imal . 


Proof  of  Proposi t ion  3.1. 

Assume  o.  =  c.  -  c'B  \\  0,  V  ;  then,  from  (3.7)  and  (3.8), 

.1  1  B  |  -  !  ’ 


1  9 


z 


=  c' B  )  5  x 

B  /L  J  1 

Note  that  -  0  for  j  corresponding  to  a  basic  variable.  Thus  any 

change  from  the  current  solution  would  result  in  an  increase  of  some 
x  (nonbasic),  and  t lie  value  of  z  would  increase  or  remain  unchanged. 

j 

Hence  no  improvement  in  the  objective  is  obtained  with  any  other 
solution. 

Q.  E.  D. 

From  Proposition  1,  we  have  an  optimality  condition  for  any 
feasible  basis; 


0.  =  C  -  c'B  *A.  ^  0  ,  Vj  .  (3.9) 

J  .)  B  j  - 


If,  on  the  other  hand ,  the  left-hand  side  of  (3.9)  was  strictly  negative, 

for  some  j  =  s,  then  increasing  x^  and  adjusting  the  values  of  the 

basic  variables  until  one  dropped  to  value  zero  (thus  replacing  a  current 

basic  variable)  would  decrease  the  objective  function,  provided  x^ 

entered  at  a  positive  level.  The  simplex  method  changes  the  basic  set 

at  each  iteration  with  the  entering  variable,  x  ,  designated  t  lie  non- 

s 

basic  variable  with  the  most  negative  relative  cost  factor,  The 

exiting  variable  is  the  first  basic  variable  to  be  driven  to  zero  as 

the  entering  variable  increases  above  zero  (assuming  nondegeneracy  and 

bounded  solutions).  The  method  terminates  with  the  current  basis  being 

optimal,  when  (3.9)  is  satisfied  for  all  variables. 

When  the  variable  x  is  chosen  as  the  entering  variable,  the 

s 

exiting  variable  can  be  determined  by  examining  the  ratios 

(b_1  b )  . 

1  for  alii,  (3.10) 


where  | B  A ^  j 
pressed  as 


0.  From  (3.7),  the  current  basic  variables  are  ex- 
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is  the  one 


l 


x  =  n_1b  -  (b_1a  )'x  . 

B  \  si  s 

Thus  the  first  variable  driven  to  zero  in  the  vector  x 

B 

corresponding  to  the  minimum  of  the  ratios  defined  by  (3.10). 

The  simplex  method  can  be  carried  ou  in  two  ways.  The  first 

way  (called  the  revised  simplex  method)  is  to  substitute  A  which  cor- 

s 

responds  to  the  entering  variable  x  for  A  which  corresponds  to  the 

s  r 

exiting  variable  x  in  the  basis  B.  With  this  substitution  both  the 
r 

new  solution  and  the  relative  cost  factors  can  then  be  calculated.  The 
second  way  is  to  pivot  in  the  augmented  matrix 


-1  0 


"cbr 


0 


B_1R 


about  the  term  ^B  RSJ  ,  where  s  corresponds  to  the  entering  vari¬ 
able  and  r  corresponds  to  the  exiting  variable.  The  pivoting  opera¬ 
tions  do  not  change  the  canonical  form  of  the  basic  variables  which 
remain  basic,  but  they  do  force  t  lie  column 


i  -e  B  A 

s  B  s 


B  1  ,\ 


to  the  canonical  form  of 


0 


L^J  • 

where  e^  is  the  unit  vector  with  a  one  in  the  r'*'  component;  this 
will  alter  all  of  t  ho  other  columns  corresponding  to  the  nonbasic 
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variables.  Note  that  once  a  feasible  basis  is  determined,  the  simplex 
method  insures  that  all  succeeding  bases  are  feasible. 

To  obtain  an  initial  feasible  basis,  phase  1  of  the  simplex  method 
adds  artificial  variables  to  (3.4)  and  solves  a  new  linear  program.  Let 
E  be  an  m  X  m  matrix  with  only  diagonal  terms,  and  let  e. .  =  +1,  1 1 
b  >  0,  and  e..  =  -1,  if  b.  <  0;  then,  the  new  linear  program  is 


Ax  +  Ev  -  b 


0,  v  0 


(3. 11) 


and  the  solution  terminates  ir  a  basic  feasible  solution  to  (3.4),  when 
the  simplex  method  is  applied  to  (3.11).  The  optimal  value  of  z  in 
(3.11)  is  zero  iff  (3.4)  is  ieasiblc. 


0.  Quadratic  Programming 


The  standard  quadratic  programming  [31  problem  can  be  stated  as 


minimize  z  =  c'x  +  x'Qx 
subject  to  Ax  _  b 


x  '  0, 


(3.  13) 


where  x  ,  En;  c  is  a  specified  n-dimensiona 1  vector;  b  is  a  speci¬ 
fied  m-dimensional  vector;  A  is  a  specified  (mvn)  matnx.and  Q 
is  a  specified  (n  >  n)  matrix.  It  is  hereby  assumed  that  Q  is  posi¬ 
tive  semidefini  t e. 

Since  problem  (3.12)  is  a  convex  programming  problem,  the  Kuhn- 
Tucker  necessary  conditions  are  also  sufficient  conditions  (or  optimaliU, 
Thus  a  solv  lion,  x,  to  the  following  necessary  conditions  is  an  optimal 

solution  to  (3.12). 


1 


u  =  c  +  2Qx  -  A '  y  >  0 

v  =-b  +  Ax  >0 

x  >0 

y  >0 

=  o,  y  v  =0,  Vi  .  (3.13) 


If  we  define 


2Q 

-A ' — 

c 

M  = 

,  and  q  = 

_  A 

0  _ 

-b 

the  necessary  conditions  may  be  written  as 


w  =  Mz  +  q 


w ,  7.  0 , 


w .  z .  -  o ,  u 

ii  '  i 


(3.1-1) 


where  M  is  p  v  p. 

Complementary  Pivot  Theory.  Problem  (3.  1‘1)  is  a  statement  of 
the  fundamental  problem  of  the  c  mpleinentary  pivot  theory  f 31 .  Although 
(3.  Id)  is  solvable  by  this  theory  for  various  classes  of  M,  l  lie  dis¬ 
cussion  here  will  be  restricted  to  M  being  positive  semide  f  i  ni  t  e,  as 
it  is  in  the  quadratic  programming  problem  (3.12). 

Note  that  we  are  looking  for  a  complementary  solution  to  the 
linear  equations  in  (3. Id),  i.e.,  a  solution  to 

w  -  Mz  +  (|  , 
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with  w  z  =0,  Vi  .  Uc  will  initiate  with  n  solution  that  is  complo- 
i  i 

mentary  but  that  may  not  bo  nonnegativc.  Wo  will  then  retain  this 
complementary  property  while  seeking  a  nonnegativc  solution. 

The  problem  in  the  structured  form  of 


(3.  15 


is  observed  with  the  transformations  being  made  b\  substituting  a  vari¬ 
able  z .  (or,  in  later  steps,  some  w  )  in  the  extreme  left  column, 
replacing  a  variable  in  the  column,  and  then  pivoting  on  the  system  of 
equations  by  changing  the  column  q  and  Lite-  matrix  M.  The  variables 
in  the  left  column  are  called  basic,  and  the  variables  in  the  row  above 
the  matrix  M  (or  M  after  traits  lorma  tion)  are  called  nonbasic.  The 

problem  is  initiated  In'  setting  w  q  and  /.  0  for  all  i.  If 

i  i  i 

any  q .  is  negative,  pick  the  w,  corresponding  to  min  q.,  and  let  it 
be  a  distinguished  variable.  The  lollowing  can  be  taken  as  a  general 
j  to rat  ion. 

Increase  the  complement  fdelined  by  (3.11)1  of  the  distinguished 
variable  and  determine'  the  blocking  variable  which  is  either 

(a)  a  basic  variable  being  driven  below  its  lower  bound  (usually 
zero)  by  an  increase  of  the  driving  variable,  or 

(b)  the  distinguished  variable  which  is  driven  toward  zero. 

f The  first  variable  to  block  in  either  (a)  or  (b)  becomes  the 
blocking  variable. 1 

If  the  blocking  variable  is  not  the  distinguished  variable, 
then  replace  t  lie  basic  blocking  variable  with  the  increasing  nonbasic 
(driving)  variable  by  pivoting  about  the  point  m  in  the  matrix  M, 

where  m  is  the  term  in  the  current  matrix  that  corresponds  to  the 

tli  rS  t  It 

s  column  (the  driving  variable)  and  the  r  row  (the  blocking  vari¬ 


able).  Now  increase  the  complement  oi  the  lormcr  blocking  variable 
(now  nonbasic)  until  a  new  blocking  variable  is  louiul. 


If  the  blocking  variable  at  any  stage  is  the  distinguished 


variable,  make  it  nonbasic  at  value  zero  and  make  the  driving  variable 
basic  (by  pivoting). 

At  this  point,  a  complementary  solution  exists.  Then  look 
at  all  q.  (determined  after  pivoting)  and  choose  the  most  negative  to 
determine  the  new  (basic)  distinguished  variable.  The  algorithm  termi¬ 
nates  when  all  q  >  0.  The  nonbasic  variables,  placed  in  the  row  above 
1  — 

the  matrix  M,  are  at  level  zero,  except  for  the  driving  variable,  at 
any  time. 

The  pivoting  rule  is:  pivoting  on  mrs> 


1 

rs  m 

rs 


m 

-  is  , 

m.  =  -  ,  Vi  A  r 

is  m 

rs 


in  . 

r.) 


-m 

_ £2 

m 

rs 


V.J  t  s 


m 

i  J 


m  . 
i.l 


m  m 
i  r  r  j 

m 

rs 


Vi  /  i' 

.i  /  s  . 


Let  q  =  m  and  apply  the  pivot  rules  given  above.  For  basic  vari- 
i  iO 

ables  that  correspond  to  negative  q .  and  not  di slingu ised ,  we  define 
their  common  lower  bound  to  be 


!'  <  min  q  ^  , 

instead  of  zero.  Tims  r  is  the  lower  bound  that  blocks  the  decrease 
of  a  basic  variable. 

It  has  been  shown  by  Dantzig  and  Cottle  [31  that  the  algorithm 
terminates  in  a  solution  to  the  quadratic  programming  problem  when  (3.14) 
has  a  feasible  solution. 


C.  Generalized  Programming 

The  generalized  programming  problem  can  be  represented  by 


Choose  a  vector,  P,  in  a  convex  set,  Cc.  E  ,  such  that 


maximize  A 


subject  to  U^A  +  Pp  =  S 


(3. 16) 


where  and  S  are  specified  n-dimensional  vectors,  and  p  is  a 
scalar.  [The  results  here  are  easily  applied  to  an  extended  form  of 
(3.16),  where  the  linear  equations  become 


V  +  Vl  +  P2L12  +  •••  +  Vq  =  s 


Pj  =  1  '  Vi  , 


and  each  P.  is  drawn  from  a  convex  set  C  .] 
l  i 

# 

Thus,  we  are  looking  for  some  vector  P  or  a  convex  combination 
i  * 

of  vector  P  ,  all  in  set  C,  so  that  the  linear  equations  are  fea¬ 
sible,  i .  e.  , 


l()>.  +  P  =  S 


l'  A  +  N  P*\  =  S 

0  l 


(3.  17) 


V,  .  =  1 

.  i 


(3. 18) 


and  the  resulting  value  of  A  is  a  maximum  over  the  choice  of  all  the 
elements  in  set  C,  which  satisfy  the  linear  equations.  Note  that,  if 
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any  set  of  P  is  in  C,  any  convex  combination  of  that  set  is  also  in 
C.  Hence  (3.17.)  and  (3.18)  are  equivalent  when 


"  I  ^ 


=1'  ' 


The  solution  procedure  assumes  we  have  on  hand,  initially,  n 
particular  choices  of  P1  t  C  so  that  the  following  linear  program 
(called  a  restricted  master) 


max  . 

i-  A 


subject  to  U?  +  p1,  +  ...  +  Pn, 

0  1  'n 


‘•1  +  •  •  • 


=  1 


i.j  _  0  (3.  19) 


has  a  unique,  feasible,  nendegcrate  solution  with  the  basis  being 
defined  as 


0 

B  = 


\  p1  p,r 

L()  1  •  •  •  1 


0  1  .  .  . 


and  being  nonsingular  (by  definition). 

V  i  0  0 


i 


i  l nee  for  each  P 


C,  P 


zl  p  V 


where 


U .  is  a  solution  to  (3.19),  is  in  C  and  is  a 


feasible  solution  to  (3,16),  but  not  necessarily  the  optimal  solution. 

0  r 

To  test  P  Land  hence,  any  solution  to  (3.16),  generated  from  a 

,  _  0 
basis!  for  optimality,  a  row  vector  is  determined  to  satisfy 


~°B°  =  (1,0 . 0)  . 


(8. 20) 
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From  n,  we  find  a  vector  P 
a  value  b  so  that 


which  is  not  necessarily  unique,  and 


_ 0—n+l  min  — 0— 

0  =  n  P  =  r;  >  , 
PcC 


(3. 21) 


where  P  = 


0,  the  current  solution  is  an  optimal  one. 


If  b  <  0,  (3.19)  is  augmented  by  P  and  t  lie  new  linear  program  is 

then  solved.  The  general  iteration  starts  with  a  solution  to  the  re¬ 
stricted  master  program 


maximize  A 

n+k 

subject  to  UqA  i  'S  p\  = 


(3.  22) 


Let  B  be  the  optimal  basis  to  the  linear  program  (3.22),  and  let 

the  dual  (optimal)  variable  to  (3.22),  be  defined  analogous  to  (3.20). 
k  a  +  k  4 1 

Then,  c  and  P  are  found  from  the  subproblem, 


-  k— n  -*  k  f  1 


nun  _k— 
0  P 


(3. 23) 


k+1  tli 

Ii  -  0,  the  solution  to  t  lie  k  iteration  of  the  master 

k+l  >i  +  k+ 1 

problem  is  optimal.  If  ''  0,  then  P  can  be  adjoined  to 

(3.22),  and  the  solution  to  (3.16)  is  improved.  The  value  i 


the  maximum  amount  by  which  the  value  of  the  current  basis  “  can  be 

i  -n  i  i  k+l 

improved.  Dins,  •  ^  constitutes  an  upper  bound  to  the  optimal 


solution  of  (3.16)  .  It  is  known  that  these  upper  bound  evaluations  can 
vary  considerably  from  one  intonation  to  the  next.  Accordingly,  the 
least  of  those  evaluations  is  saved  from  all  t  t  <  rations,  including  the 


current  one. 


It  has  been  shown  that,  if 
to  (3.19)  is  nondegenerate  (p 


C  is  bounded  and  the  initial  solution 

_ -X 

^  0)  ,  it  -»  ii  and  P  — »  P  [where 


n+k 


i  =1 


and  p.  is  a  solution  to  (3.22)],  on  a  subsequence  k,  and  that  P 

_ # 

is  optimal  for  (3.16).  n  satisfies  the  properties 


* 

P 


* 

n  /  0 

(3.24) 

“  P  >  “  P  =0,  for  all  P  t  C  . 


Moreover,  if  C  is  a  polyhedral  set,  then  the  subproblem  (3,23)  is  a 

linear  program,  and  t  lie  iterative  process  terminates  in  a  finite  number 

of  steps.  It  should  be  noted  that,  in  any  case,  the  objective  function 

improves  with  each  iteration,  and  a  feasible  solution  always  exists  to 

the  master  problem.  Also,  the  initial  solution  (or  columns)  for  (3.19) 

can  be  obtained  by  a  procedure  simitar  to  a  phase  I  simplex  method. 

Remembering  that  the  usual  form  of  a  generalized  program  includes 

the  sum  of  the  vectors  P.  ^  C  ,  where  the  C  are  convex  sets,  the 

i  t  i 

vector  S  need  not  be  fixed,  but  it  must  be  drawn  from  a  convex  set  ,  ^  . 
Thus  the  generalized  program  becomes 

max 

A 

P,  S 

r  -•  p;  -  S-  =  o 

o 


•.  =  1  ,  (3.25) 

where  P  t  C  and  S  & .  In  this  case,  the  subpmblem  is  extended  to 


k 


| 

i 

1 

I 

5 


find 


as  in  (3.23)  and 


This  dual  is  the  equivalent  of  finding  a  particular  hypcrplane  to  support 

the  convex  set  C.  If  a  solution  to  the  dual  is  known,  then  a  solution 

* 

to  tlie  primal  may  bo  found  using  the  dual  solution,  ,  to  find  the 
¥ 

vectors,  P  <.  C,  that  satisfy 


*  -k 
If  P  is  unique  and  the  primal  has  a  solution,  P  must  be  the  solu- 
¥ 

tion.  If  P  is  not  unique  and  the  primal  lias  a  solution,  then  some 
convex  combination  of  all  the  P  must  lornt  the  primal  solution. 
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Chapter  IV 


PARAMETRIC  PROGRAMMING 

In  this  chapter,  algorithms  are  presented  for  solving  parametric 
linear  and  quadratic  programming  problems,  where  the  dependence  on  the 
parameter  is  nonlinear  and  occurs  only  in  the  linear  part  of  the  objec¬ 
tive  function.  These  parametric  programming  problems  arise  in  the  sub¬ 
problem  of  the  generalized  programming  formulation  of  the  optimal  control 
problems . 

A.  Parametric  Linear  Programming 

We  consider  the  following  problem  linear  in  x 

■K 

find  x  (t)  to 
minimize  /  (  t  )  '  ,\ 
subject  to  Ax  =  b 
x  _  0 

t  v  r Tj ,  T,;i  ,  (i.i) 

and  the  following  problem  quadratic  in  x 

■¥ 

f  i  nil  x  ( t  )  to 
minimize  /  ( 1 )  '  x  t-  xQx 
subject  to  Ax  b 
x  _  0 

t  <  r  Tj  ,  Tvl  .  (1.2) 

In  both  of  the  above  cases,  A  is  a  given  m  ■  n  real  matrix,  b 
is  a  given  n-d i mens i ona 1  vector,  Q  is  an  n  x  n  positive  semidefinile 
matrix,  x  is  a  vector  in  E1',  and 

/<  • )  I/,  (i ) .  /.  (  t  ) .  /..( t )  !  (  1.3) 

i  K 
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is  a  given  vector  valued  function,  each  component  of  which  is  a  solution 
to  some  homogeneous,  linear  differential  equation  with  constant  real  co¬ 
efficients  that  may  depend  on  k.  Such  /  (t)  are  of  the  form 

k 


S  p„<»  «Sk,t 


(4.  4) 


where  p  ,(t)  is  a  polynomial  with  real  coefficients  of  degree 
k  i 

so  that 


V  m  .  =  N 

kl 


and  s  are  constants  so  that ,  if  s  is  complex  for  i  odd, 
ki  ki 

s  is  its  conjugate  and  p,  .(t)  -  p  .  (1).  It  follows  llicn  thnl 

ki+1  ki  ki+1 

these  /  (t)  are  real-valued  functions  of  t. 
k 

The  lemmas  and  theorems  that  follow  are  required  to  show  convergence 
of  the  proposed  algorithm. 

Lemma  4.1.  If  /  .  ( t )  is  a  solution  to  a  homogeneous  linear  differential 
equation  with  constant  real  coefficients  of  order  N  and  il  for 
some  t  =  t  _ 


/><•> 


fl  ,  V 

7TT  /  •  ' 


dN  1  r /  /til 


t  hen  /  y ( t )  0  for  a  1 1  t 


Proof  o i  Lemma  4.1. 

/  (t)  solves  in  equation  of  the  form 


dt 


/*(l)  =  V*(°  +  ai 


d;,(«) 

dt 


.  +  a 


N-l 


,N-1  .  N 

d  /  *<  t  > 


dt' 


(4.5) 


At 


t 


0’ 


d"  ,  * 

“S  |  =  0  .  (4.6) 

dt  I.  . 


By  taking  the  derivative  of  both  sides  of  (4.5)  and  substituting  (1.6), 


_d 

dt 


N+l 

N+l 


,(  t  ) 


t  =1 


0 


0  . 


If  this  procedure  is  continued,  all  derivatives  of  /,(•)  at  t  =  t 

become  zero.  Therefore,  with  /.,(')  =0  and  all  of  its  derivatives  at 

zero  for  t  =  tQ  and  with  /  +  (t)  being  able  to  expand  (at  t  =  t  )  to 

a  Taylor  series,  /  (t)  must  be  constant  and  have  value  zero  for  all  t 
* 

Q.K.l). 

Definition  4.1.  A  vector  y  is  said  to  be  lexicographically  greater 

than  zero,  il  at  least  one  component  is  non-zero  and  the  first  such 
component  is  positive;  this  vector  can  bo  denoted  as 


>  c-  ()  . 

A  vector  y  is  lexicographically  greater  than  a  vector  /., 

v  A-  z 

if  y  -  z  0.  A  vector  is  said  to  be  lexicographically  greater  or 
equal  to  zero,  if  it  is  lexicographically  greater  than  zero  or  equal 


to  ze r< i  . 


A  similar  definition  is  true  for  one  vector  to  be  lexicographically 
greater  or  equal  to  another  vector. 


1 


Thus  f' (t  )  N  0  and,  by  using  Lemma  A.  2,  a  o  does  exist  for 
t  <  t  <  t  +  b,  so  that 


J-1 


J-1 


/JO  =  <  f(‘)  =  “T  T  /  JO 


dt 


j-1  '*  0 


dt 


.1-1  '* 


From  Taylor's  theorem,  it  is  known  that  a  number  between  t 

and  t,  exists  for  any  given  t,tQ  <  t  <  t  +  c  and 


0 


/,<»  "  /.<>„>  * 


(j-i) : dt 


Since 


,1-2 


/^(10)  dt  /*(tQ) 


dt 


.1-2  '  *  0 


(t  )  =  0 


and 


/.<■)  11 


i-1 


dt 


J-1 


j-1  '  0 


dt 


(  t  )  =  0  , 


/JO  _  0.  Hence,  it  follows  that  /JO  U  for  all  t  ft  ,t  +  b)  . 

Q.  E.  D. 


1  he  first  algorithm  presented  here  is  based  on  the  simplex  method 
and  solves  problem  (1.1). 

Find  x  (t)  to 
minimize  /  (  t  )  '  x 

subject  to  Ax  =  b,  x  0,  t  t  fT^.T  1  . 
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Here,  /(t)  is  a  vector  whose  i  component,  / ^  ( t ) ,  is  a  real  scalar 
function.  At  time  t  ,  let  BQ  be  an  optimal  basis  for  the  linear 
program 


minimize  / ( t  )  '  x 
subject  to  Ax  =  b 

x  >  0  .  (4.7) 

Let  the  solution  be  x  =  x  (tQ).  Since  x  (t  )  is  a  feasible  solution 
to  Ax  =  b,  x  >  0,  it  remains  a  feasible  one  to  problem  (4.1)  for  all 
t,  but  it  is  not  necessarily  an  optimal  solution.  Thus  how  the  opti¬ 
mality  test  for  0^  varies  as  t  takes  on  the  values  t  =  tQ  +  t' , 
where  c  >  0,  must  be  investigated.  Let 


*  'k<l)  -  'n  <" 


<4.  8) 


where  A 

k 


is  the 


th 


column  of  the  matrix 


A, 


=  inf  '  J/  (t  +  c,  B  )  <  0)  ,  and 

K  U  U 


(4.  9) 


=  min  4. 

0  k  k 


(4. 10) 


It  is  possible  that  the  ,  presented  above,  is  zero.  7^(1,  nQ )  i  -s 

the  relative  cost  factor  for  any  t  of  column  k  when  B^  is  chosen 

as  the  basis.  Then  the  ordering  of  columns  A  can  be  taken  so  that 

A  . . . ,  A  correspond  to  the  m  columns  of  B  , 

1  m  d 

For  any  basic  variable  x  ,  associated  with  the  optimal  basis  B^, 


/  ( t , B  )  -  /  ( t )  -  / ,  ( t )  u"1 A .  =/.(<)  -  /  .  ( t )  -  0,  Vt 

'  l  0  i  B  0  i  l  l 


(4. 11a) 


And,  for  t  =  t  , 


'VW  -°-  Vk  ’ 


(4.  lib) 
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with  equality  being  held  when  k  corresponds  to  a  basic  variable. 

Therefore,  the  solution  to  (4.1)  remains  optimal,  i.e.,  it  satisfies 

/  ( t , B  )  >  0  when  t  t.  [t  ,t  +  co^  f°r  some  N  given  by  (4.10). 

R  0  0  0  0  0 

Let  /'^(t,D)  be  the  j1*1  derivative  (with  respect  to  t)  of  the 

relative  cost  vector  / ( t , B)  when  B  is  the  basis  under  consideration. 

Let  /^(t,B)  be  the  component  of  the  above  vector  corresponding  to 
k  1 

column  k  of  A.  Let  A  be  the  new  linear  programming  matrix  ob¬ 
tained  after  deleting  all  columns  (variables)  for  which  the  relative 

cost  factors  /  (t  ,B  )  =  7?(t  ,B  )  are  strictly  positive  for  t  =  t 
K  0  0  k  0  0  o 

A  general  iteration  is  given  with  L(J  -  T  ;  a  flow  chart  of  the  algorithm 
follows  the  iteration. 


Step  I:  Solve  the  linear  program 


minimize  • ( t  )  'x 
subject  to  Ax  b 

x  _  0  (4.12) 

to  obtain  the  optimal  basis  BQ.  If  the  solution  is  unique  at  t+ 

(i.e.,  all  relative  cost  factors  for  nonbasic  variables  are  strictly 
positive),  proceed  t  ■>  Step  III. 

If  the  solution  s  non-unique,  lix  t  -  t  and  proceed  to  Step  II, 

0 

starting  with  ’  =  i  and  A  =  A. 


Slep  II:  Let  the  matrix 


columns  of  A' 


-1 


be  composed  ol  the  matrix  Bj-i  and  all 

-j-1 


having  the  relative  cost  (actors 


s  t 


"o'Vd 


where  ‘(t^.B^  refers  to  the  (j-1)'  derivative.  To  simplify 


notation  in  (4. IS)  below,  let  the  new  x  any 

J 


vee t  ors  cor  respond i ng 

to  A,J  also  be  denoted  by  x  and  ,  although  they  are  now  shortened 

Then,  solve  the  linear  program 
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x  and  /  vectors. 


Let  EL  denote  the  optimal  basis.  [Computationally,  it  is  convenient 
to  start  with  the  previously  optimal,  basic  feasible  solution  corre¬ 
sponding  to  j-1  and  then  to  apply  the  simplex  method  to  obtain  an 
optimal  solution  to  (4.13),]  If  the  solution  to  (4.13)  is  unique  or  if 
j  =  N  -  1,  use  the  optimal  basis  B.  and  proceed  to  Step  III. 

If  the  solution  is  non-unique  and  .j  <  X  -  1 ,  increase  j  by  1 
and  repeat  Step  II. 


Step  III:  Using  the  optimal  basis  from  Step  I  or  II  in  place  of  B^  in 
(4.8)  and  (4.9)  for  all  columns  k  (optimal  basic  columns  can  be  ignored 

since  their  t  =  +«)  find  Then  calculate  «.  .  The  solution, 

*  _1  '  '  0 
x  (t)  -  B  b,  is  then  optimal  for  all  t  ^  ft^, t  +  Moreover,  it 

will  be  shown  that  >  0. 

If  *o  +  L0  N  T2’  '  *’e  Parnmetric  programming  problem  is  solved, 

However  if  the  solution  is  not  reached,  repeat  t  lie  general  iteration 
with  t  =  tQ  +  k.q  replacing  tQ. 

That  this  algorithm  does  terminate  in  a  finite  number  of  steps  to 
a  solution  of  (4.1)  for  all  t  remains  to  be  shown.  The  re¬ 

mainder  of  this  section  is  devoted  to  showing  a  finite  number  of  steps 
t o  t he  so  1 ut i on. 


Lemma  4.3.  If  the  relative  cost  (actors  for  some  basic  H  are  zero  for 

a  subset  of  columns  S  and  positive  lor  the  remaining  columns  T, 

then  the  same  is  true  for  any  basis  1!  whose  columns  are  in  S. 

1 


Proof  of  Lemma  4.3. 

The  vector  of  coefficients  of  the  objective  equation  of  the  original 

matrix  can  be  replaced  bv  the  relative  cost  vector  lor  1!  .  The  price 

0 

of  simplex  multipliers  relative  to  |!()  sat  is  lies  B  - 
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0 


FI  .OK  CHART  FOR  PAR  AMr'TR  I C  I,  I  NK  \R 


PROGRAMMING  AI.GOR  I  TPM 


1 

and  that  of  B 


1 


0, 


columns  of  BQ  and  B 


since  the  objective  coefficients  for  both 

0  1 

are  now  zero  by  hypothesis.  Hence  "  =  -  =  0, 


and  it  follows  t  ha  t  their  relative  cost  factors  are  identical. 


Q.  E.  D. 


Theorem  4.2.  At  t  lie  end  of  the  general  iteration,  the  following  vector 
relations  are  satisfied: 


,  (  t  .Ill 
'  k  0 


—  /  (t  ,B) 
tit  '  k  0 


>-  0 


N-l 


dt 


N-l  '  k  0 


for  k  -  1 . n 


or 


l)[,  (  t  ,  11)1  -  0  for  all  k 

k  fl  , 

where  I)  is  the  final  basis  on  terminating  the  iteration  of 
Step  II  at  l  . 

Proof  of  Theorem  1.2. 

For  k  corresponding  to  tile  basic  variables  of  (  1,11), 

1)[;  (t  ,B)|=  0.  For  the  basis  B,  let  the  basic  variables  be  x  , 
'  k  0  B 

and  let  the  nonbasic  variables  be  x  .  'Ihe  problem  is  separated  as 


=  b 


z 


B’  '  R 


=  0  . 


(4.  14) 


By  pivoting  on  the  m  +  1  rows  of  (4.14),  t  lie  first  m  columns  are  now 
unit  vectors,  and  the  system  becomes 


I  i  R 


b 


7.  - 


where  /  is  t ho  relative  cost  factor  vector  of  the  nonbasic  variables. 
U  is  separated  into  two  matrices,  and  R^,  so  that  the  relative 

cost  factors  corresponding  to  the  columns  of  R  are  zero,  and  those 
corresponding  to  the  columns  of  Ro  are  negative.  Then,  the  problem 


rows 
1 . m 


XR. 


row 

m+1 


'VV”  *  0 


V 


L 


11 


row 
m+2  ■ 


0 


is  observed,  where  only  Clie  variables  correspond  in;;  to  the  columns  of 
Kj  and  I  are  allowed  to  enter  the'  basis. 

According  to  Lemma  4.3,  the  pivoting  procedures  of  the  simplex  algor¬ 
ithm  retain  the  zero  elements  ol'  row  in  +  1  at  level  0  and  the  positive 
elements  of  row  in  +  1  are  at  positive  values  for  every  stage;  these 
piocedures  term  ante  with  all  relative  cost  factors  of  row  m  +  2,  corre¬ 
sponding  to  I  and  K  ,  at  nonnegntive  values.  At  termination,  because 
of  the  simplex  method  stopping  rule,  a  new  set  of  basic  variables  is  found 
having  the  property  of  the'  component  >f  the  basic  variables  in  rows  m+  1 
and  m  +  2  being  at  zero  (after  pivoting);  the  components  of  the  nonbasic 
variables  are  either 

(1)  zero  in  row  in  +  1  and  nonnegative  in  row  m  +  2,  for  vari¬ 
ables  corresponding  to  columns  of  1  or  It  in  (4.15)  or 

(2)  strictly  positive  in  row  m  +  1,  for  variables  corresponding 
to  columns  ol  l{<;) . 

If  the  variables  are  as  in  (1)  above,  the  ones  having  zero  components 
in  row  m  +  2  are  chosen  with  their  columns  for  consideration  in  the 
next  stage  of  the  algorithm.  Once  a  nonbasic  variable  has  a  positive 
relative  cost  factor  at  any  stage  ,  it  can  be  assumed  that  its  rela¬ 
tive*  cost  factors  were  at  zero  in  previous  stages  also;  lienee  it  can  no 
longer  enter  tin  basis.  Since  the1  relative  cost  (actors  in  the  first 
k  -  1  stages  can  never  change  sign  by  pivoting  in  tin*  k  *  and  later 
stages,  its  derivative  vector  must  be  lexicographically  greater  than  ()  . 
Thus  at  completion  ol  the  N  stages,  all  ill  rivalive  vectors  must  be 
lexicographically  greater  than  or  equal  to  zero. 

Q.K.D. 


Theorem  4  .3  .  The  basis  13,  obtained  at  the  end  ol  Step  II  in  the 

algorithm  for  any  l  L  ,  remains  optimal  for  the  interval 

It  ,  t  +  !  ,  where  is  strict  1\  positive  . 

0  0  0  t)  '  1 

12 


Proof  of  Theorem  ‘1  .3  . 


Since  each  >  (t,B)  is  a  member  of  the  class  of  solutions  to  homo- 
k 

geneous,  constant  coefficient,  linear  differential  equations,  c  is 

K 

strictly  positive  by  Theorem  4.1.  By  the  definitions  given  for  t  and 

K 

t  ,  the  basis  B  satisfies  the  optimality  criteria  for 


t 


It  ,  t 
0  0 


Q  .  E  .  D  . 


Theorem  4.4.  The  number  of  basis  changes  in  any  finite  interval  [T^,T  ] 
is  a  finite  number,  and  the  parametric  programming  problem  is 
solvable  in  a  finite  number  of  steps. 


Proof  oi  Theorem  4 .4 . 

At  anv  switching  point  t  ,  there  exists  a  basis  B  and  an 

0  p 


•-  >0,  so  that  B  is  optimal  for 

P  „  P 


t  ,  t  +  •.  ) 
0  0  p 


There  also  exists 


a  basis  B 


a  nd  a  n 


>  0,  making  B  optimal  for  t.  d  t 


0 


'V' 


It  follows  then,  if  there  is  a  cluster  point  at  t  ,  then*  would  be  an 


infinite  increasing  sequence  ol  switching  points  t.  e(t^ 


which  could  be  bypassed  by  a  single  switch  at  any  such  t.  to  basis 
•¥- 

B  .  This  establishes  the  existence  of  a  finite  number  of  basis  changes 

in  anv  finite  interval  t  >  It  ,T  ! . 

1  2 

What  remains  to  be-  shown  is  that  the  algorithm,  as  presented,  solves 


the  parametric  programming  problem  in  a  finite  number  of  steps.  As  dis¬ 
it 

cussed  above,  Let  us  assume  there  is  a  switching  point  t  It, 


and  a  switch  from  basis  B 


i-l 


i  0 

to  B  .  To  simplify  the  discussion,  let 


us  also  assume  that  B  di tiers  from  B 

i  i-l 


by  the  introduction  ol  tine 


incoming  column  k,  and  that  the  value  of  the  incoming  variable  x 

0  k 
x  >  0,  i.e.,  the  basic  solution  is  nondegenern t e . 

k 

The  optimal  value  ol  the  objective  /  in  the  neighborhood  of  i  , 

i 

takes  the  form  of 


zo(t-Vi) 


■(t,B.  ) 

k  i-l 


i  r  t 


i  i 


t 

i 


4  3 


Both  z„(t,B  , )  and  >,  (t,B  . )  can  bo  shown  to  bo  linear  combinations 

0  l-l  k  l-l 

of  solutions  to  homogeneous,  linear  differential  equations  with  constant 
real  coefficients  and  hence  they  themselves  are  also  solutions.  In  addi¬ 
tion  for  t+,  >  (t+,B  )  v.  0  otherwise  there  would  have  been  no  switch 

i  k  l  i-l  g  nd 

from  B.  to  B..  This  means  that  the  vector  of  the  0,1  ,2  . 

s  t  — 

N-l  order  derivatives  of  >(t,B  . ),  evaluated  at  t.,  is  lexi- 

k  l  -1  l 

cographica 1 1 y  negative.  It  follows  then  that  the  function  z  is  discon¬ 
tinuous  in  t  at  t  .  in  at  least  one  of  its  j1*1  order  derivatives.  On 

i  ^ 

the  other  hand,  the  optimal  value  of  z ,  in  t  lie  interval  t  tTt  -  t  ,  t^l  , 
can  also  be  expressed  as 


and  is  continuous  in  all  derivatives  at  t.,  which  is  a  contradiction. 

The  above  argument  is  now  extended  to  the  case  where  B  ^  is  as¬ 
sumed  to  differ  from  B  bv  several  incoming  columns.  The  term 

i 

x  >  (t,B.  )  is  then  replaced  by  a  sum  of  terms,  each  of  which  is  lexi- 

k  k  i-l 

cographically  negative  at  t  =  t.,  hence  their  sum  is  lexicographically 
negative  and  t ho  discontinuity  at  t  =  1.  follows.  (Note  that  1  lie  de¬ 
generacy  of  basic  solutions  in  the  simplex  algorithm  is  assumed  to  be 
handled  by  the  standard  right-hand  side  lexicographic  rules  of  the 
simplex  method. ) 

Since  eacli  basis  change  is  accomplished  by  a  finite  (at  most  N) 
number  of  linear  programs,  the  parametric  programming  problem  is  solv¬ 
able  in  a  finite  number  of  steps. 

Q.  K.  I). 

Corollary  d .  1 .  The  solution  x  (t)  to  (•!.  1)  is  a  piecewise  constant 
vector  function  of  t  with  a  finite  number  of  discontinuities. 

Proof  of  Corollary  1.1. 

d 

Since  x  (t)  is  constant  for  t  ^ft  ,t^  +  .  ^1,  this  follows  im¬ 
mediately  from  Theorem  i.  l. 

Q  K.  I). 


H 


1 


Proposition  -1.1.  When  all  the  s.  for  each  ^  ( 1 )  are  real,  the  upper 
bound  on  the  number  of  switchings  in  an  infinite  time  interval  for 
an  (tn  v  ni)  matrix  A  is 

(n  -  m)  N  . 

Proof  of  Proposition  1.1. 

For  any  /^(t)  that  is  a  member  of  the  class  of  solutions  to  an 
N^*1  order  homogeneous,  constant  coefficient,  linear  differential  equa¬ 
tion  and  for  real  and  distinct  s  ,  it  is  known  [11  that  the  function 

t 

/^(t)  has,  at  most,  N  roots  of  /  +  ( t )  =  0. 

'  11  \ 

At  the  most,  there  are  |  possible  bases  for  A;  for  each  of 

\  »>/ 

these  base; ,  there  are  (n-m)  nonbasic  variables.  For  eaclt  nonbasic 
variable  x.,  the  relative  cost  factor  / . ( t , b)  has  at  most  N  points 
at  which  it  crosses  the  value  zero  anti  thereby  creates  a  possible  basis 
swi tch. 

Q.  E.  n. 

13.  Parametric  Quadratic  Programming 

'I'lie  conditions  for  which  the  parametric  programming  problem  lias  a 
solution  are  found  in  this  section.  Also,  an  algorithm  based  on  the 
complementary  pivot  theory  procedure  lor  quadratic  programming  is 
constructed. 

The  quadratic  programming  problem 

1  i nd  x  (  t  )  to 
m i n i m i ze  /  ( t )  '  x  +  xQx 
subject  to  Ax  I) 

x  0  , 


(1.  Hi) 


can  be  formulated  in  the  complementary  pivot  theory  as, 


¥  ¥ 

find  w  (t),  z  (t),  so  that 


w  =  mz  +  q ( t ) 


w  z.  =0,  Vi 
i  i 


w  0,  z.  0,  Vi  ,  (‘1.17) 

i  —  l  — 


where 


2Q 

-A' 

/(t) 

M  = 

,  q(t)  = 

A 

0 

-b 

“  x  ~ 

u  ” 

z  - 

,  and 

w  = 

V  _ 

where  y  is  the  vector  of  final  variables  to  the  quadratic  programming 
problem,  and  u,  v  are  slack  vectors  of  the  necessary  conditions  for 
quadratic  programs,  as  was  stated  in  Chapter  III.  The  necessary  con¬ 
ditions  in  (4.17)  for  Q  positive  semidefim  to  are  sufficient  at 
any  t  =  t^.  From  t  lie  results  of  Dantzig  and  Cottle  [til,  the  complemen¬ 
tary  pivot  theory  algorithm  terminates  in  a  solution  to  (4.17)  when  M 
is  positive  semi de f i n  i  t e,  providing  the  solution  set  for 

w  =  Mz  +  q,  w  0,  z  0, 


is  nonempty. 

To  show  that  the  above  solution  set  is  nonempty,  a  solution  must 
be  shown  to  exist  for  every  t  in  t  lie  parametric  programming  problem. 

If  such  a  solution  does  exist,  it  must  satisfy  Hit*  conditions  of  (4.17). 
Therefore,  let  us  assume  the  absence  of  unbounded  solutions,  L.c.,  that 
the  set 
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X  =  { x|  x  >  0,  Ax  >  b] 


is  bounded  and  nonempty. 

Proposition  4.2.  The  parametric  quadratic  programming  problem  has  a 
solution  for  every  point  t  ctT^.T^l,  when  X  is  nonempty  and 
bounded,  ;  ( t )  is  a  vector  function  with  each  of  its  components  bounded 
in  tlie  interval  [T^,Ty],  and  Q  is  positive  semidcf  ini  te. 

Proof  of  Proposition  4.2. 

Since  the  objective  is  continuous  in  x  over  a  compact  set  X,  it 
attains  its  infinuin  at  a  point  in  X. 

Q.  E.  D. 

Propos i t ion  4,3.  Given  the  above  conditions  on  /It),  X,  and  Q,  the 
form  w  =  Mz  +  q,  w,z  0,  w  z.  =  0  has  a  solution  for  every 
t  f  T  ^ ,  T  g  ] ,  and  this  solution  can  be  found  by  using  t  lie  methods  of 
complementary  pivot  theory  in  a  finite  number  of  pivot  operations  on  M. 

Proof  of  Proposition  4.3. 

As  stated  above,  Dantzig  and  Cottle  [  d  I  have  shown  that  the  comple¬ 
mentary  pivot  theory  algorithm  converges  to  a  solution  of  the  quadratic 
programming  problem,  it  the  solution  set  lor  w  Mz  +  q ,  u,z  _  0,  is  non¬ 
empty.  Since  X  is  nonempty,  and  since  the  infinuin  of  ;(t)'x  +  x'Qx 
is  attained  in  X  for  each  t,  then  the  necessary  and  sulfieienl  con¬ 
ditions,  i.e.,  a  solution  to  (1.17),  must  exist.  Thus,  the  conditions 
lor  termination  of  the  algorithm  are  satisfied. 

Q.  K.  I). 


(emember  l  ng  the  characteristics  <>j  /  (  L )  explained  earlier,  we 

will  now  show  that  onl\  a  linite  number  ol  solutions  are  considered  when 
solving  the  parametric  programming  problem  lor  all  t  in  the  linite 
interval,  [T^.T^l. 
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Theorem  1.5.  There  are  a  finite  number  of  changes  in  t  lie  set  of  basic 


variables  of  (4.17)  for  the  parametric  programming  problem  over  a 
finite  interval. 

Proof  of  Theorem  4.5. 

Let  the  M  matrix  be  (n  v  n) .  Since  each  complementary  solution 
has  n  basic  variables,  there  are,  at  most,  (  ",'1  )  complementary  solu¬ 
tions.  By  using  the  pivoting  procedures,  the  characteristic  of  any 
solution  is 


~  _  n 
w 

" 

z 

= 

q  ( t ) 

+ 

M 

7. 

-  _ 

w 

where  w  is  the  set  of  basic  variables  of  the  vector  w, 


and  x,  the  set  of  basic  variables  of  the  vector  x.  The  solution,  for 
any  L ,  i s 


- 

-  - 

r  i 

w 

- 

q(  t  ) 

0  , 

/. 

/. 

. 

w 

The  vector  function  q(l)  is  <>l  t  lie  form  /(t),  and  each  component 


q(t)  has  a  finite  number  < *  1  zero  crossings  in  any  finite  interval  o) 


Let  t  lie  so 

points  be  labelled. 

1 

i  ’ 

.  ki 
i 

for  t  he 

.  t  h 

i  compl emeu  tar 

sol ut ion. 

The  se  t  of  points 

o 


y 


t . 


IK 


1 


are  the  only  points  at  which  the  set  of  basic  variables  can  change  in 
the  finite  interval.  The  sot  T  is  countable  and  has  measure  zero.  If 


the  points  are  ordered,  i.e. ,  t^, t  ,...,t  ,  then  a  set  of  basic  vari- 

0  1  h 

ables  that  remain  basic  must  exist  for  any  interval  [ t  . . t  This  is 

.1  j+1 

true  because  a  solution  has  been  shown  to  exist  for  every  t  t[t  ,t 

J  J  +  1 

thus  tlie  set  of  basic  variables  cannot  change  in  this  interval. 

Q.  E.  D. 


At  t h i s  point,  an  algorithm  is  presented  to  provide  a  basic  solution 
to  the  complementary  problem;  this  solution  remains  optimal  over  a  posi¬ 
tive  interval.  The  method  used  does  not  require  prior  knowledge  of  the 
switching  points  and  does  not  assume  nondegenerate  solutions  at  these 
points.  The  algorithm  employs  the  same  pivoting  procedures  of  the  com¬ 
plementary  pivot  theory  algorithm  presented  in  Chapter  III. 

Definition  l..'l.  We  will  say  a  vector  y,  which  is  lexicographically 
smaller  than  zero  (i.e.,  -y  0)  lexi co-increases  to  y,  if 
(y  "  >')  ?■  0,  and  if  the  component  <  i  ~  corresponding  to  the  first 
nonpositive  component  ol  y  becomes  nonnega t i ve  . 


Def  ini  t  ion  ■!  ■  We  will  define  a  lex  i  co-minimum  of  a  set  of  vectors  as 

that  vector  y  where  all  the  other  vectors  in  it  are  lexicograph¬ 
ically  greater  than  or  equal  to  \  . 


Do  f  i  n  i  t  ion  1  ■  5  .  For  every  t^,  !+-  (t  )  is  do  lined  as  the  vector 

i 


w 


d<)  (11 


dt 


d*  '  i|  (  O 


dt 


\-t 


If) 


where  q.ft^)  is  the  current  value  of  the  i  component  of  qO^) 
under  the  pivoting  procedures  (i.e.,  those  procedures  leading  to 
a  particular  complementary  solution).  Let 

t  -  inf  Mi|.(t„  +  t)  <  0]  , 

1  l  0 

and  let  =  min  c  .  . 

0  j  t 

We  will  now  present  an  algorithm  for  solving  t ho  parametric  quad¬ 
ratic  programming  problem.  It  should  be  noted  that  this  algorithm  is 
also  applicable  to  the  parametric  linear  programming  problems  discussed 
in  Section  IV. A;  however,  this  method  is  more  complicated.  A  flow  chart 
and  proof  of  the  algorithm's  finite  termination  will  follow  the  general 
iteration  given  here  for  any  t  =  t^. 

Step  I:  (a)  Solve  the  complenu  n t a rv  pivot  theory  problem 

w  Mz  +  C|  (  t  ) 

w  z  0 ,  w  ,  z.  0  , 
i  t  - 

for  positive  semidef i p i t e  matrix  M,  and  (b)  examine  the  nonnegative 
complementary  solution  for  t  t  ;  i.e.,  is 

1>—  ( t  )  h-  0  or  —  0  ? 

q  o 


If  D-^r  (I  )  < —  0  or  equal  to  zero,  proceed  to  Step  III;  if  it  is  not, 

M  i  0 

go  to  S t  ep  II. 

Step  II:  Choose  an  index  i  so  that  Ilq  (t())  is  a  minimum  over  all 
tlie  derivative  vectors  that  are  lexicographically  less  titan  zero.  Retain 
all  other  variables  of  those  variables  which  have  derivative  vectors  lex¬ 
icographically  less  than  zero,  at  their  present  lower  bounds,  or  let  them 
in' rcnsc  (when  forced  to  decrease  they  are  blocking  variables).  Then  let 
the  basic  variable  corresponding  to  i  be  the  (  istinguished  variable  and 
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■Kiid^rwitMl inintHiHiit^.fcau. 


Let  t 


TERMINATE 


Solve 

w  =  Mz  + 

q(to) 

W  Z  - 

i  i 

0 

w  >  0 

z  >  0 

Is  ^-V 


Replace  with  t  + 


Are  all  D—  ( t rt)  0 

C1t  °  —  YES 

or  =  0? 


Find  c  ,  Vi 
and  fc- 


Pick  i  so  that  Dq-.O^)  is 

lexico-minimum  and  lot  i^1 

basic  variable  be  distinguished. 

Use  complementary  pivot  theory 

algorithm  to  increase  Dq-.  (t^) 

while  keeping  all  variables  ,} , 

so  that  q.(t  )  =0  at  their 
.)  0 

lower  bound. 


Pi'occed  until  distinguished  variable 
lex i co-i ncreases  (by  leaving  basis 
at  level  zero). 


FLOW  CHART  FOR  PARAMETRIC  QUADRATIC 
PROGRAMMING  ALGORITHM 


perform  the  standard  complementary  pivot  algorithm  with  the  new  lower 
hound  restrictions;  terminate  when  the  distinguished  variable  lexico¬ 
graphically  increases  (i.e.,  when  it  drops  out  of  t  lie  batis  by  becoming 
level  0).  The  solution  of  Step  I  is  used  to  initiate  Step  II.  Now, 
return  to  ( b)  of  Step  I  with  the  current  solution. 

Step  III:  I'sing  the  previous  definitions,  calculate  ,  .  for  all  i 

and  t  ,  the  minimum  of  •_  .  The  final  basis  at  t  is  then  optimal 
0  i  0 

for  t  t  r '  Q.  t  +  <  ()1  .  If  a  solution  for  t  t  ^  is  desired,  re¬ 

turn  to  (b)  of  Step  1,  using  the  solution  at  t  «.  ..nd  proceed  with 

on 

the  algorithm. 

The  procedure  given  here  provides  a  solution  to  the  parametric 
quadratic  program  for  any  interval  oi  the  parameter. 

It  remains  to  be  shown  that  each  step  in  the  algorithm  does  termi¬ 
nate  in  a  finite  number  of  executions  and  that  the  final  basis  generated 
at  any  t  is  optimal  over  a  linite  positive'  interval  for  the  quadratic 
program.  Hence  the  remainder  id  th  s  section  is  devoted  to  this  proof. 
Step  1  is  solvable  iii  a  finite  number  "I  steps,  if  the  problem 

w  M/  +  q(l  ) 
w  ’  y.  0 ,  w ,  y  0 

can  be  solved.  Since  it  is  known  that  the  quadratic  program  has  a 
solution  for  every  t,  we  are  assured  < •  t  the  complementary  pivot  theory 
algorithm  converging  to  an  optimal  solution  in  a  I  ini  to  number  o(  steps. 

Proposition  1.  I.  Stop  II  must  torminate  with  :i  complementary  solution 
having  t  !,<•  derivative  vector  ■  >(  the  distinguished  variable  lexi¬ 
cographically  increased  while  o  i  in*  i  variables  at  their  lower  hound 
are  not  1  e\ i co-dec  reused. 


Proof  of  Proposition  1.1. 

The  complementary  p.vot  theory  algorithm  when  initiated  with  a 
basic  (  compl emeu t a rv)  solid  t  u  ,  terminates  in  another  complementary 
solution,  since  we  assume  leusibilitv  o|  the  quadratic  program. 


because 


t ho  distinguished  variable  and  its  complement  are  converging  to  a  non¬ 
negative  complementary  solution,  this  termination  occurs  when  the  dis- 
inguished  variable  leaves  the  basis.  When  it  leaves  the  basis,  the 
variable  has  a  derivative  vector  equal  to  zero;  thus  it  has  lexico¬ 
graphically  increased.  All  other  variables,  at  their  lower  bounds,  were 
not  permitted  to  lexico-decrease;  all  variables  entering  the  basis  are 
permitted  to  increase  only.  Thus  the  new  solution  has  no  variables 
lexicographically  less  than  zero,  and  the  distinguished  variable  is 
lex i co- i ncr eased. 

Q.  E.  D. 

Theorem  4.6.  The  algorithm  terminates  in  a  finite  number  of  steps  to  a 

complementary  basis  that  is  optimal  over  a  finite  positive  interval. 

Proof  of  Theorem  4.6. 

The  execution  of  Step  II  lexi co-i ncrease  at  least  one  of  the  vari¬ 
ables  that  was  lexicographically  less  than  zero  while  not  lexico- 
decreasing  any  of  them;  and  it  does  not  introduce  any  now  variables 
lexicographically  less  than  zero.  Since  there  are  only  a  finite  number 
of  lexicographically  less  than  zero  vectors  (at  most,  n)  and  since 
eacli  has,  at  most,  n  components,  the  execution  of  Step  II  must  termi¬ 
nate  after  at  most  n  times  with  each  execution  requiring  a  finite 
number  of  steps.  The  termination  condition  is  that  all  basic  variables 
are  lexicographically  greater  than  or  equal  to  zero  (all  nonbasic  vari¬ 
ables  lexicographically  equal  to  zero).  Thus  by  Theorem  4.1,  the  basic- 
variables,  and  hence  the  solution  to  (  1.17)  are  nonnegative  over  a 
finite  positive  interval. 

Q.  K.  I). 

From  the  above  results,  it  lias  been  determined  that  a  finite  number 
of  basis  changes  are  required  to  solve  the  parametric  quadratic  program¬ 
ming  problem  for  a  finite  interval  of  the  parameter.  The  solution  to 
the  parametric  quadratic  program  need  not  be  piece-wise  constant  as  it  is 
in  the  parametric  linear  programs.  For  each  interval  where  a  single- 
basis  remains  optimal,  1  he  solution  will,  in  fact,  havo  the  character¬ 


istic  form 


where  w  and  z  are  the  complementary  variables  in  their  original  form, 

and  z  and  w  are  their  complements.  q(t)  is  nonnegative  over  the 

interval  and  has  the  form  where  each  of  its  components  solves  some  par- 
til 

ticular  N  order  homogeneous  constant  coefficient  linear  differential 
equation.  When  the  value  of  some  q^(t)  goes  negative  for  some  t, 
the  basic  variables  w,  z  are  no  longer  optimal,  and  a  new  set  of  basic 
variables  must  be  found  with  the  complementary  property.  The  new 
values  of  q^(t)  are  just  linear  combinations  of  the  former  components. 


Chapter  V 


;  J™ 
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GENERALIZED  PROGRAMMING  ALGORITHM  FOR 
OPTIMAL  CONTROL  PROBLEMS 

The  mathematical  programming  results  obtained  in  the  previous  two 
chapters  are  applied  to  the  linear  system,  continuous-time  optimal  con¬ 
trol  problems  to  formulate  [9]  a  generalized  linear  program.  A  solution 
procedure  based  on  this  formulation  is  then  developed  and  is  shown  to 
terminate  in  an  optimal  solution,  i.e.,  an  optimal  control  is  provided 
to  the  continuous-time  problem. 

A.  Formulation 

The  control  problems  will  now  be  formulated  as  generalized  programs 
and  then  the  subproblems  will  be  shown  to  be  parametric  programming  prob¬ 
lems  of  the  form  presented  in  Chapter  IV.  The  control  problem  can  be 
restated  as 


min  J  =  f  xn(t)  dt  =  x  (T) 

u ( • )  Jo  ° 


x(t)  =  Fx(t)  +  Gu(t) 


_n  ni 

x  e  E  ,  u  e  E  ,  and 


(5.1a) 


(5.1b) 


xo 

.  x  . 


x(0)  e  ,  and  x(T)  e  ST  , 


u(  t)  e  U  =  (u|  Au  >  b}  ,  U  c  E  , 


F  is  an  n  x  n  real  matrix,  and 
G  is  an  n  X  m  real  matrix, 


m 

xQ(t)  =  fQx(t)  +  gQu(t)  +  h  ^  +  u^t)'Qu(t)  (5.1c) 

1=1 

where  Is  a  fixed  real  n-vector  and  g^  Is  a  fixed  real  m-vector, 

h  is  a  real  constant,  and  Q  is  an  n  X  n  real  matrix. 

We  will  first  consider  strictly  linear  cost  functionals,  I.e., 
h  =  0,  Q  =  0. 


Letting 


F  = 


and 


G  = 


g0 


(5.2) 


the  completely  linear  system  can  be  expressed  by 


x( t)  =  Fx( t)  +  Gu( t)  . 


(5.  3) 


When  a  particular  vector  function  u  (t)  and  an  initial  condition 
x*(0)  are  given,  the  solution  to  (5.3),  at  t  =  T,  is 


x(T)  =  e 


TF 


A 

x1 ( 0)  +  / 

•4) 


e(T"t)F  Gu1  ( t)  dt  , 


(5.  4a) 


and  the  solution  to  (5. lb)  is 


x(T)  =  e 


TF 


A 

x1(0)  +  / 

Ja 


e(T-t)FGui(t)  dt  . 


(5. 4b) 
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F  m 

The  sot  Sg  c  E  is  defined 


F  TF 

S0  =  (y|x  e  Sq>  y  =  e  x] 


If  Sq  is  a  convex  set,  Sq  is  also  a  convex  set  by  the  linear  mapping. 
The  set  i!  c  E°  can  be  defined  by 


*  '  ST  -  S0  ■ 


i  -?z|z  =*  -y,  x  €  ST,  y  €  , 


Proposition  5.1. 


If  S^  and  Sq  are  convex,  then  4  is  convex. 


Proof  of  Proposition  5.1. 

,  1  2  F  12 

Let  x  ,  x  be  points  in  S  and  let  y  ,  y  be  points  in  S  ; 
1  i  2  2  U  T 

then  x  -  y  ,  and  x  -  y  are  in  4.  For  all  A,  0  <  A  <  1, 

A(x1-y1)  +  (l-X)(x2-y2)  =  Ax1  +  ( 1 -A)  x2  -  Ay1  -  (1-A)  y2  . 

1  2  F  F 

Ax  +  (1-A)  x  e  Sq  since  SQ  is  convex. 

1  2 

Ay  +  (1  -?\)  y  o  S^  since  S^  is  convex. 

11  2  2 

Thus  A(x  -y  )  +  (1-A)(x  -y  )  e  4,  and  implies  that  4  is  convex. 


Q.  E.  D. 


Let  Sq  and  S^  be  defined  as 


-  n+1  -  J0 

s0=<ytE  y  y 


y0  =  o, y  e  s0 


c  (_  »♦!  -  T  y0  1  \ 

y  =  M'  y»'°'y£Sy} 


the  sets  S  and  £  are  similarly  defined. 
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Using  the  above  set  of  definitions,  we  can  restate  the  initial  and 
final  state  constraints  of  the  fixed  time  control  problems  as 


x(0)  =  0 


xQ(0)  =  0  ,  and 


x(T)  e  A  . 

Thus,  it  can  be  assumed,  without  loss  of  generality,  that  the  system 
initiates  at  the  origin  with  no  prior  costs. 

1,  Control  Problems  Formulated  as  Generalized  Programs 

If  we  take  the  vector  functionals  of  the  control  P  =  P[u(t)] 
to  be  defined  by 


and 


P  =  j  e(T“t)F  Gu ( t )  dt  , 


l 


P  =  P[u(t)]  to  be  defined  by 


(5. 5a) 


P  =  /  e(T_t)F  Gu( t)  dt  , 


-jC 


(5.5b) 


then,  let 


J 


C  =  [  P 


u ( t)  el',  P 


-/ 

Jo 


e<T-,)FOu(t)  dtl 


and 


oJp 

1 1 


u(t)  t  U  , 


p  =  i  e(T_t,F  Gu(t)  dt  | 


-L 


Proposition  5.2.  If  U  is  a  convex  set,  the  set  C(C)  is  convex. 


Proof  of  Proposition  5.2, 

1  2 

Let  u  (t),  u  (t)  be  vector  functions  in  U,  for  all  t,  and 
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*  •  I'ljy'd  • 


'  KiTr7  7 


mmtmmwmm 


■  :  ■&  ■$:  > .:  , mm ■  p r -u-mm : 


p1  =  r  e(T-t)r  Gu^(t)  at  c  c 


Mo 

M1 


P2  =  /•  e(T-t)FGU2<t)  d,  €  C 


For  all  A,  0  <  A  <  1,  A  =  (1-A), 


Xp1  +  Xp2  =  X  f  .(T-t)F  G»x(t)  dt  +  X  fT  e<T-,)F  0u2(t)  dt 


(T-t)F  „P,  .  t.2 


G[Au  (t)  +  Au  (t)]  dt 


1  —  2 

Since  U  is  convex,  Au  (t)  +  Au  (t)  £  U,  Vt. 


-  „.  2 

Thus  AP1  +  AP  e  C.  and  C  is  convex. 


Q.E.D. 


Remembering  that  che  state  at  time  0  is  assumed  to  be  at  the 
origin  and  using  the  definitions  of  P,  C,  and  Eq.  (5.4b),  we  find  C 


is  equivalent  to  R^,  the  reachable  set  of  U  at  time  T.  The  control 


problem  is  feasible,  iff 


c  n  &  t  0 


Given  a  specified  control  function  u(t.),  the  cost  associated  with  that 


control  is  J[u(t)],  since  the  cost  at  time  t  =  0  is  zero.  Note  that 


[':]■ 


where  pQ  =  J[u(t)]  , 


when  u(t)  is  the  control  generating  P  by  Eq.  (5,5a).  Thus 


v  V 


_  1 
c  =  < 

P  e  En+1|  p  = 

- 1 

«  J3 

1 

,  p  =  P(u)  , 

PQ  =  J('J)  , 

u(t)  e  U 

1 

L  *  J 

Note  also  that  by  using  Proposition  5.2,  C  is  convex. 

Let  Up  =  (1,0, . . , ,  0)  and  note  that  the  first  component  of 
the  vectors  S,  in  the  set  rf,  is  defined  to  be  zero.  Also  note  that 

the  first  component  of  the  P  vector  represents  the  cost  of  using  the 
control  (and  its  corresponding  trajectory)  generating  P.  Thus  we  are 
looking  for  a  vector  P  e  C,  a  vector  function  u(t)  generating  P, 
and  a  vector  Serf  to  satisfy 

_max_  A  ,  pi  v  >  0 
P  e  C 

subject  to  UQA  +  Pp  =  Sy 

M  =  1 

v  =  1  ,  (5.6) 

where  p  and  v  are  scalars.  Maximizing  A  is  equivalent  to  minimiz¬ 
ing  J[u(t)],  the  first  component  of  the  vector  P,  where  u(t)  gen¬ 
erates  P.  Since  P  must  be  taken  from  a  convex  set  C  and  S  must 
be  taken  from  a  convex  set  rf,  the  above  formulation  is  a  generalized 
program  of  the  Dantzig-Wol fe  type.  In  the  following  chapter  we  will 
show  that  an  optimal  solution  to  the  control  problem  is  an  optimal  solu¬ 
tion  to  the  generalized  programme  *g  problem.  We  will  now  show  that  any 
solution  to  the  generalized  programming  problem  is  an  optimal  solution 
to  the  original  control  problem. 

A  solution  to  the  generalized  programming  problem  consists  of 
a  vector  P  in  the  reachable  set  RT,  a  control  function  u(t)  in  the 
admissible  control  region  U  generating  P,  and  a  vector  S  in  the 
constraint  set  of  terminal  states  rf,  ,o  that 

P  =  S  . 
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The  above  equality  insures  the  transformation  of  the  system  from  an 
initial  point,  x(0)  e  to  a  final  point  x(T)  e  by  the  vector 
function  u(t),  chosen  from  U.  Thus  it  is  a  feasible  control.  By 
minimizing  J  over  all  feasible  sets  of  P  and  S,  we  can  find  a 
feasible  solution  with  the  least  cost.  This  is  precisely  an  optimal 
solution  to  the  continuous-time  control  problem. 

2.  Generalized  Programming  Subproblems  as  Parametric  Programs 

To  complete  the  generalized  programming  formulation,  its  sub¬ 
problem  must  be  described.  Here  we  assume  there  are  at  least  n  +  2 
vectors  P1  and/or  S1  available  to  provide  a  feasible  solution  to 
(5.6),  so  that  the  problem 

max  A  ,  u.  v  >  0 

(Ji  V  ~ 


V  + 


M1 + 


+  PJ 


=  S 


j+1 


Qj  +  P 

+  o  y 


+  M2  +  •••  +  Mj  =  1 

Vi  +  v9  +  •  •  .  +  v  =  1  ,  (5.7) 

l  6  P 


is  solvable  and  has  a  dual  solution  vector 


where  it'  =  (it^, .  .  .  ,  it  ) ,  The  subproblem  is  then  formulated  in  two  parts: 


min  it' 
S  6  J 


S 

0 

1 


and 


(5.8) 


P 

min  it'  1 
P  €  C 


(5.9) 


0 
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be  an  m-dimensional  vector  function.  Thus  the  subproblero  becomes 

Find  u(t)  e  U  ,  t  e[Cl,T] 

so  that  7(t)*u(t)  is  a  minimum  .  (5.13) 

From  (5.12),  it  is  obvious  that  7(t)  has  the  property  of  each  of  its 

s  t 

components  being  a  member  of  the  class  of  solutions  to  an  n  +  1  order, 
homogenous,  constant  coefficient,  linear  differential  equation.  Since 
our  attention  is  restricted  to  those  U  that  are  polyhedral  sets,  (5.13) 
becomes 

min  7(t) 'u(t) 

Au(t)  >  b 

t  e[0,T]  .  (5.14) 

(Note  that  the  inequality  may  be  reversed  or  an  equality  without  loss  of 
generality.)  Thus  a  solution  u(t),  for  the  subproblem,  can  be  obtained 
by  using  the  parametric  linear  programming  methods  of  Chapter  IV. 

In  a  similar  manner,  we  can  formulate  minimum  fuel,  minimum 
time,  and  quadratic  loss  in  control  problems  as  generalized  programs.  It 
can  also  be  shown  that  the  minimum  fuel  and  minimum  time  are  special 
cases  of  the  linear  loss  problems  just  described.  Since  generalized  pro¬ 
gramming  can  be  applied  to  general  convex  programming  problems,  we  can 
formulate  optimal  control  problems  with  loss  functions  convex  in  the  con¬ 
trol  variable  as  generalized  programs.  However,  only  the  quadratic  loss 
in  the  control  case  will  be  c1'  cussed  in  detail,  since  this  is  (to  the 
author's  knowledge)  the  only  general  nonlinear  convex  loss  function  which 
has  a  known  finite  solution  procedure  for  the  parametric  subproblem.  Sep¬ 
arable  piecewise  linear  (convex)  functions  of  the  control  can  also  be 
formulated  as  a  special  case  of  the  linear  loss  problem,  although  it  will 
not  be  shown  here. 


The  previous  formulations  can  be  generalized  if  we  observe 
following  general  linear  system  control  problem  and  use  the  notation 
given  in  Chapter  II, 


min 
u(t)  e 


-j(S 

u  */o 


(t)  dt, 


where  x(t) 


xQ(t) 

x(t) 


„n+l  , 

e  E  ,  and 


c(t)  =  Fx( t)  +  Gu( t)  +  f(u)  UQ  ,  (5.15) 

,  linear  loss 

where  f(u)  =  ^  £  |u.|  ,  minimum  fuel 

u'Qu  ,  quadratic  loss  , 


where  Q  is  positive  semidef inite,  u(t)  e  U,  t  e[0,T],  x(0)  e  S^, 

x(T)  e  S  ,  and  SQ,  ST  are  convex  sets  in  E  .  We  note  that  J  is 
a  convex  functional  in  u(t),  since  u(t)  is  a  vector  sequence  drawn 
from  a  convex  set  U,  and  f(u)  is  a  convex  function  in  u.  Thus  the 
solution  to  Eq.  (5.15)  can  be  noted  as 


x(T)  =  e™  x(0)  +  fT  eF(T_t)  Gu(t)  dt  +  C  eF(T_t)  f[u(t)]  UA  dt  . 

Jo 


Jo 


0 


(5.16) 


Now,  we  define  P  and  P,  as  before,  by 


F(T-t) 

e 


Gu(t)  dt  , 


and 


(5.17) 


P  = 


P 


F(T-t) 

e 


Gu(t)  dt  + 


f[u(t)]  dt  UQ  , 


(5.18) 
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where  UQ  is  an  n  +  1-dimensional  unit  vector  with  a  one  in  the  first 


component.  Thus  the  second  integral  of  (5.18)  becomes 


I  y  |u  ( t ) | dt  ,  minimum  fuel  , 


i=l 


rT  u(t>- 

Jo 


Qu(t)  dt  ,  quadratic  loss 


Also  as  was  done  before,  we  can  now  define  C  as 


)  —  n+1 

r  z 

F(T-t)  .  . 

/  P  e  E 

P  = 

/  e  Gu(t)  dt 

)  r\ 

•/o 

z  >  J[u(t)]  ,  u(t)  e  U,  Vt1 


Thus  the  P,  as  defined  in  (5.18),  are  members  of  C.  We  also  note 
that  C  is  a  convex  set,  since  J[u(t)3  is  a  convex  functional. 

The  vectors  S  and  set  &  are  defined  as  before.  Thus  the 
general  linear  system  control  problem  can  be  formulated  as  a  generalized 
programming  problem, 


Find  P  t  C  ,  S  e  &  ,  to 


max  A  ,  p.v  >  0 
P  e  C 


subject  to  UqA  +  P|j  =  Sy 


u  =  1 
v  =  1 


(5. 19) 


Again  we  note  that  the  solution  to  the  generalized  program  is 
one  ir  which  a  vector  P  is  found  so  that  the  last  n  elements  P 


belong  to  the  set  S,  and,  out  of  all  possible  vectors,  P  t  ,  i.e., 
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feasible  solutions  to  the  control  problem;  the  first  component  of  P  , 

taken  to  be  the  value  of  the  loss  functional,  is  minimal.  Thus  the 

*  _ * 

vector  control  function  u  (t)  generating  P  is  a  solution  to  the 
optimal  control  problem. 

We  now  describe  the  subproblem  corresponding  to  the  minimum 
fuel  and  quadratic  loss  problems  as  in  Eqs.  (5.3)  and  (5.9).  For  the 
general  case,  following  the  similar  reasoning  given  for  the  linear  loss 
case,  the  subproblem  to  the  restricted  master  problem  becomes,  (remember¬ 
ing  that  n  =  1,  since  (11^0,0)  is  a  basis  vector  and 


It' 


=  1  ) 


min  f  Jf[u(t)]  +  (rt  n  ' )'  e(T_t)F  Gu(t) 

u(-)  Jo  { 


dt  +  jt 


n+1 


(5.20) 


Defining  7(t)  as  before,  (5.20)  becomes 


min  7(t)'u(t)  +  f[u(t)] 
u  ( t ) 

subject  to  u(t)  e  U  ,  Vt  .  (5.21) 

For  the  quadratic  loss  problem,  with  U  =  (u|Au  >  b,  u  >  0) , 
(5.21)  becomes  a  parametric  quadratic  programming  problem  in  u(t)  of 
the  form  discussed  in  Chapter  IV  [due  to  the  form  of  /(t)]. 

We  now  look  at  the  minimum  fuel  problem  for  two  classes  of  U. 
The  first  class  has  the  classical  form  of  the  minimal  fuel  problem, 
where 


U  =  (u||u.(t)|  <  1,  i  =  1 . m)  , 


and  the  second  class  is  a  general  polyhedral  U. 


For  the  first  case,  (5.21)  has  the  following  solution  (the 
singular  arcs,  y(t)  =  1  for  an  interval,  are  not  discussed  because  no 
solution  is  defined), 


<  1  , 

ui(t) 

=  0 

7t(  t) 

<  -1  , 

u^t) 

=  1 

7±(t) 

>  1 

u1(t) 

=  -1 

for 

all  i  . 

(5. 22) 

The  magnitude  of  one  as  a  bound  for  the  control  is  noted  to 
be  nonrestrictive,  since  G  can  be  scaled  to  permit  other  values.  The 
formulation  can  also  be  adapted  to  treat  lower  bounds  on  u^(t)  with 
magnitudes  that  differ  from  the  upper  bounds.  These  changes  affect  the 
ranges  of  7 ^ ( t )  in  (5.22).  Thus  the  subproblem  -r'r  the  standard  mini¬ 
mum  fuel  problem  has  a  well-defined  solution,  and  its  execution  in  re¬ 
lation  to  the  master  problem  is  proportionately  as  quick,  regardless  of 
the  size  of  the  control  space. 

The  minimum  fuel  problem  for  general  polyhedral  sets  U  has 
a  subproblem  equivalent  to  that  of  the  linear  loss  case.  Following  the 
same  steps  given  above,  the  subproblem  for  general  U  which  replaces 
(5.23)  is 


min 

u(0 


7(t)u(t)  + 


2 


u.(t) 

1 


subject  to 


Au(t)  ">  b 


0  <  t  <  T  .  (5.23) 

This  is  equivalent  to  a  parametric  linear  programming  problem  of  the 
type  presented  in  Chapter  IV  when  the  variable  udt),  which  is  un¬ 
restricted  in  sign,  is  replaced  by  the  difference  between  two  nonnega¬ 
tive  variables, 
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(5.24) 


u. ( t)  =  u  (t)  -  u. (t)  . 
ill 

The  constraints  are  replaced  by 

Au  -  Au  >  b 

u  >  0 

u  >  0  .  (5.25) 

Since  linear  programming  algorithms  consider  only  basic  solutions, 

and  cannot  be  basic  at  the  same  time  because  their  columns  A^  and 

-A  ,  respectively,  are  linearly  dependent.  Thus  for  every  i,  either 

u.  <."  u  must  be  at  level  zero.  Using  this  result,  |u  (t)|  can  be 
i  l  1  l  1 

replaced  by 

I u . ( t )  |  =  u.(t)  +  u.(t)  ,  (5.26) 

1  l  1  l  i 

and  the  equivalent  parametric  linear  program  is 

min  ^  [)\(t)  +  1]  u^(t)  +  ^  [-/^(t)  +  1]  u.  (t) 

u,u  i  1  1  i 

subject  to  AU(t)  -  Au(t)  >  b 

u.  (t) ,  u.(t)  >  0 

i  l  — 

for  0  <  t  <  T  .  (5. 21) 

Since  a  generalized  programming  formulation  is  shown  to  be  used 
for  finding  a  feasible  solution  to  the  linear  system  control  problems, 
the  minimal  time  problem  can  be  solved  with  these  methods.  The  subprob¬ 
lem  to  the  generalized  programming  problem  for  feasible  solutions  is  also 
shown  to  be  a  parametric  linear  programming  problem  of  the  type  discussed. 
Here  we  will  present  a  solution  procedure  (similar  to  the  one  proposed 
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by  Ho  [11]) to  the  minimal  time  problem  that  is  equivalent  to  finding, 
if  it  exists,  a  feasible  solution  to  the  generalized  programming  form¬ 
ulation  of  linear  systems  and,  of  course,  to  determining  whether  or  not 
a  feasible  solution  exists. 

The  minimal  time  problem  can  be  stated  as 

ft 

min  I  dt  =  t 
u(-)  -JQ  f 

(t  is  free) , 

where  x(0)  is  given  and  x(tf)  =  0,  and 
x(t)  =  Fx(t)  +  Gu(t) 

x(t)  c  En,  u ( t )  e  U  c  Em  .  (5.28) 

Thus  we  wish  to  find  the  shortest  time  in  which  we  can  transfer  the  state 

from  a  given  point  to  the  origin  or,  equivalently,  to  find  the  minimum 

time  for  which  a  control  u(t)  t  U  exists  to  transfer  the  system  from  a 

given  point  to  the  origin.  The  latter  statement  is  the  one  related  to 

a  generalized  programming  formulation  of  the  minimal  time  problem.  De- 

TF 

fine  P  as  before,  for  any  fixed  T,  and  S  =  -e  x(0)  .  Let  the 
set  of  all  P  be  C  h  R  ,  i.e.,  the  reachable  set,  which  is  convex. 

We  want  to  know  whether 

S  t  Rt  ,  for  any  fixed  T  ,  (5.  29) 

so  that  we  can  find  the  minimum  T  for  which  (5.29)  holds.  Equiva¬ 
lently,  we  want  to  know  whether  a  solution  exists  for 


Pp  =  s 

M  =  1 

P  e  C  =  ,  for  any  T  ,  (5.30) 
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and  again  we  want  to  find  the  minimum  T  for  whica  a  solution  to  (5,30) 
exists.  The  solution  procedure  for  (5.30)  for  any  T,  will  be  discussed 
later.  At  this  point,  we  will  solve  the  minimum  time  problem  by  choosing 
a  T,  and  try  to  solve  (5.30).  If  a  solution  exists,  decrease  T  and 
proceed;  if  one  does  not  exist  then  increase  T  and  proceed.  If  the 
increments  for  the  increase  and  decrease  of  T  are  chosen  wisely,  this 
procedure  will  converge  to  an  answer  to  the  minimum  time  problem. 

The  solution  to  (5.30)  is  a  phase  I  generalized  programming 
procedure  that  is  also  used  to  find  initial  feasible  solutions  to  the 
optimal  control  problems  discussed  here.  A  finite  convergence  procedure 
is  shown  for  phase  I  methods  when  its  existence  is  known,  and  a  test  for 
its  existence  will  be  presented  for  control  problems  for  which  the  exis¬ 
tence  of  feasible  solutions  is  not  assumed. 

To  generalize  and  summarize  the  above  results,  the  following  class 
of  control  problems  may  be  formulated  as  generalized  programming  problems: 

min  J  -I  f(x,u)  dt  , 
u(.) 


x  =  Fx  +  Gu, 

x(0)  t  S  ,  x(T)  e  ,  u(t)  t  U  ,  0  <  t  <  T  ,  (5.31) 

o  T  —  — 

where  S^,  S^,  and  U  are  convex,  and 

f(x,u)  =  f1(x)  +  fgCu) 

where  f  (x)  is  linear  in  x  and  independent  of  u,  f  (u)  is  convex 
in  u  and  independent  of  x.  When  S^,  S  ,  and  U  are  polyhedral 
sets  and  f2(u)  cluaclra tic  in  u  or  the  sum  of  the  absolute  value  of 

the  components  (with  linear  terms  permitted),  the  generalized  program¬ 
ming  problem  is  solvable  by  the  methods  presented  in  the  previous  two 
chapters.  The  rest  of  this  chapter  is  devoted  to  the  development  of  the 
algorithm  for  solving  these  generalized  programming  problems  and  to 
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pointing  out  the  specific  features  of  the  algorithm  so  that  it  can  be 
adopted  for  special  purposes,  including  the  determination  of  feasibility 
or  its  existence. 

B.  Solution  of  the  Control  Problem 

The  first  step  and,  at  times,  the  major  problem  in  the  solution  of 
the  control  problem  is  to  find  an  admissible  control  yielding  a  feasible 
solution.  An  important  characteristic  of  the  generalized  programming 
solution  of  optimal  control  problems  is  that,  at  every  stage  in  the 
optimization  phase,  a  feasible  solution  is  always  available.  With  this 
feasible  solution,  a  bound  on  the  value  of  the  optimal  objective  func¬ 
tion  can  then  be  computed.  Thus,  if  the  solution  procedure  is  inter¬ 
rupted  before  its  convergence  to  an  optimal  solution,  a  feasible  solution 
can  be  recovered  and  an  estimate  of  how  close  it  is  to  an  optimal  solu¬ 
tion  provided.  This  estimate  or  bound  can  be  used  to  terminate  the  al¬ 
gorithm,  since  suboptimal  solutions  having  an  objective  value  close  to 
the  optimal  one,  are  generally  sufficient  for  decision  purposes.  Al¬ 
though  the  general  solution  to  the  linear  control  problems  may  be  an 
infinite  convergent  process,  the  generation  of  a  feasible  solution,  if 
interior  solutions  exist,  is  a  finite  process,  and  the  generation  of  a 
suboptimal  solution,  as  close  as  desired  in  objec  ive  value  to  the  op¬ 
timal,  can  also  be  obtained  in  a  finite  number  oi  steps.  The  algorithm 
and  its  variants  are  presented  in  this  section  along  with  convergence 
and  finiteness  proofs.  The  characteristics  of  solutions  and  their  re¬ 
lations  to  known  results  in  control  theory  are  presented  in  Chapter  VI. 

1.  Generation  of  a  Feasible  Solution 

There  are  two  major  aspects  of  finding  a  feasible  solution. 

The  first  of  these  is  the  determination  of  whether  or  not  a  feasible 
solution  exists;  t he  second  is  to  generate  the  feasible  solution,  if  it 
does  exist.  In  both  cases,  a  piiase  I  procedure  of  the  generalized  pro¬ 
gramming  problem  is  used.  The  control  problem  to  be  considered  is 

Find  a  u(  t )  t  lT  =  f  u  e  E  "  Au  <  b]  ,  Vt  , 
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and 


such  that  x(0)  e 

x(T)  e  ,  when  x  is  controlled  by 

x(t)  =  Fx(  t)  +  Gu(t)  .  (5.32) 

Without  loss  of  generality,  we  let  =  0  and  S  =  <J. 

Let  us  assume  that  the  reachable  set  R  is  a  continuum.  Let 
us  also  assume  that,  if 

Rt  n  <4  t  0  , 

then  Rt  O  &  is  a  continuum.  This  condition  insures  the  existence  of 
a  finite-dimensional  neighborhood  in  that  set  of  desired  final  states 
in  the  reachable  set.  As  will  be  shown,  these  conditions  imply  that  the 
phase  I  portion  of  the  generalized  programming  formulation  of  the  control 
problem  terminates  with  a  feasible  solution  in  a  finite  number  of 
of  steps. 

Since  the  convergence  (Chapter  III)  of  the  generalized  program 
assumes  that  a  nondegenerate  feasible  starting  solution  is  available, 
the  phase  I  procedure  must  terminate  with  such  a  nondegenerate  feasible 
solution.  This  implies,  for  an  n-dimens ional  state  space,  that  a  col¬ 
lection  of  n  +  1  vectors  P  must  be  generated  so  that  the  initial 
basis, 


is  nonsingular.  Also  a  value,  S  t  <4,  must  be  provided  so  that  B  ^ S 
is  a  vector  and  is  strictly  positive  in  all  components. 

Although  convergence  proofs  require  the  results  of  the  follow¬ 
ing  phase  I  procedure,  more  efficient  variants  (to  be  presented)  should 
be  used  in  practice  and  can  provide  a  feasible  solution  in  fewer  steps 
for  most  problems. 
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We  will  now  show  the  procedure  for  finding  a  feasible  solution 

,  n  . 

when  <5  is  a  single  point  and  a  ball  of  radius  p  in  E  (a  p- 

neighborhood)  is  also  contained  in  the  reachable  set,  R^.  The  p- 
neighborhood  is  used  to  avoid  degeneracy  problems,  in  much  the  same  man¬ 
ner  as  a  simplex  lexicographic  methods  by  perturbating  the  original  right- 
hand  side.  Thus  we  seek  to  generate  a  set  of  n  +  1  vectors  p1 ,  to 
provide  a  .ondegenerate  solution  to  the  set  of  linear  equations. 


+  P 


n+1 


n+1 


S 


^1 


+ 


+ 


Mn+1 


1 


p.  >  0  ,  Vi  . 


(5. 33) 


a.  Some  Properties  of  Convex  Sets 

Definition  5.1.  The  convex  hull  of  a  set,  X  q  E*1,  is  the  intersection 
of  all  convex  sets  in  En  containing  X. 


Definition  5.2.  The  convex  hu’l  A  of  a  finite  set  of  n+1  points, 

x  ,  x  ,  ...,  x  ,  in  E*1  is  an  n-dimensional  simplex,  if  the 
1  £  11+ 1. 

flat  of  minimal  dimension  containing  A  has  dimension  n.  The 
points  x^  are  called  vertices. 

Lemma  5.1  [12].  If  A  is  an  n-dimensional  simplex  witli  vertices 

x  (i  =  1,  ...,  n  +  1),  then  A  consists  of  all  points  x  t  e'1 

i 

for  which  constants  ^  exist,  so  that 

i 

n+1 

X  =  Y  O  X.  ,  No  =l 

A-  i  i  ^ — '  i 

i  =1 


O  >  0  . 
i  — 


73 


Definition  5,3.  A  set  of  k  +  1  points  in  En  is  geometrically  in¬ 
dependent,  if  no  (k-1) -dimensional  hyperplane  contains  all  the 
points. 

Definition  5.4.  A  set  (x  ,  x1 . xn}  of  vectors  in  En  is  point- 

wise  independent  (algebraic  counterpart  of  geometrically  indepen¬ 
dent),  if  the  k  vectors,  -  xQ,  xg  -  xQ,  xk  ~  x0  are 

linearly  independent. 

Lemma  5.2  [13].  If  X  =  {x0,  x  ,  ....  xfc)  is  a  pointwise  independent 

set  in  En,  then  there  exists  a  unique  k-dimensional  hyperplane 

k  k 

H  containing  X  having  the  property  that  x  e  H  ,  iff 

k  k 

"  ■  I  Vi  ■  X  - 1  ■ 

1=0  i=0 

where  the  Of  are  unique.  The  CY^  are  the  barycentric  coordi¬ 
nates  of  x  with  respect  to  X, 

Let  us  look  at  the  convex  hull  of  the  set  of  n  +  1 
points  in  En,  X, 


and  denote  it  by  convX.  These  points  are  pointwise  independent,  thus 
their  convex  hull  is  a  simplex. 
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Let 
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L(n+1) 


• 


(5  .35) 


If  we  take  inner  products  with  all  the  x  ^ ,  j=l .  n,  and  sum  the 


left-  and  right-hand  sides  of  the  result  (5.35),  we  obtain 


?*•  ■  i  =1  ■ 


(n+1) 


t  y  *i ' 
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£ 


* 


which  implies  1  <  2n/(n+l)  ,  which  is  a  contradiction  for  n  >  0. 
Thus  the  l 
dependent. 


Thus  the  vectors  are  pointwise  independent  or  geometrically  in- 


Q.E.D. 


Theorem  5.1.  ""he  point  x  is  in  the  interior  of  convX' 


Proof  of  Theorem  5.1. 


ConvX'  forms  a  simplex,  since  the  points  x^  are  pointwise  in¬ 


dependent  and  form  an  n-dimensional  hyperplane.  Thus 


X*  -  £  a;*;  , 


and 


i=0 


I  q;  - 


(5.  3) 


has  a  unique  solution  in  the  ex'  . 

i 


We  must  now  show  that  >  0,  Vi.  Notice  that 

l 


1=0 


or 


I  h*;  -  t^t  ° 


i=0 


(5. 37) 


Since  x'.  =  x.  +  ,  (5.37)  becomes 

li  l 


I 


i=0  ■- 


/  4  \ 

- 

(or.  -  -i- 

x.  +  O'1.  Ax 

\  i  n+1l 

i  l  i 

_ 

=  0 


(5. 38) 
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By  taking  the  inner  product  of  (5.38)  with  each  x^,  i  =  1, 
cessfully,  we  get  the  set  of  equations, 


...  n  suc- 


or,  in  general 


n+1 


I  a;<*v  v  - 


i=0 


“j 


-sti*  I  x2>  - 


i=0 


<r 

n 


_  J_  +  y  a\{£at\,  X  )  = 
n+1  /.  1  1  n 


i=0 


aj 


1 

n+1 


I 

i=0 


x  ) ,  j 


=  1.  ....  n 


Since 


(Ax!,  x  >  <  - x  , 

(n+1) 


a1  >  — 

i  n+1 


(n+1) 


(n+1) 


i=0 


(n+1) 


Thus  Ct\  >  0  for  i 

l 


=  1 ,  . . . ,  n .  Now, 


i=0 


Li. 


and  we  take  the  vector  product  of  (5.38)  with  1  to  get 
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1 


Ik 

i=l  ' 


st)  + 1 

'  i=0 


which  implies  by  using 


1  °i  = 


1  -  ao  > 


V  —  =  !  -  — 

nj.1  n  4- 1 


n+ 1  , 


1  +  — +  \  o  '.  (£x! ,  i; 

n+l  l  l  — 


-i-  +  N  rr'.(£*!,  i) 

n+l  i  l  — 

i  =0 


Remembering  that  ||Zj<.|]  <  l/(n+l)  and  _1||  =  \'n,  we  obtain 


0  n+l 


liL-  V  O' 
2  ,  _  1 
(n+l)  - 


(n+l)' 


Thus  >  0,  and  x  is  in  the  interior  of  convX', 


Q.  E.  1). 


By  using  the  assumption  that  a  .  -neighborhood  about 


S  =  x  is  also  in  R  ,  the  following  set  of  points  are  found  in  R  , 
T  T  ! 


y  -  Tf 


79 


p  (  T  n+1  \  n+1  /  -  T  n+1  V  ’  * . .  /  ’ 


where  e  ,  i  =  1,  .  .  .  ,  n  is  an  n-dimensional  unit  vector  with  a  one  in 
th  1 

the  i  row  and  e.  is  the  null  vector.  Geometrically,  X  defines 

u  p 

the  vertices  of  an  n-dimensional  simplex  with  X  defined  by  the  bary- 

centric  coordinates  of  l/(n+l) ,  for  each  point  in  X  . 

P 


Let 


X.  =  - 

i  T 


1 

P 

n+11 

in+1 

)  1  +  A 

I  -  n+1 


Vi 


Proposition  5.6. 
for  any  i, 


9 

Let  a  ball  of  radius  [p/(n+l)]  [l/(n+l)  ]  ,  about  XA 

be  N . ,  then 

l 


Proof  of  Proposition  5.6. 


X,  -  = 

i  T 


N 

i 

1 

/  P 

n+1 

l  n+1 

1 

f  J±. 

n+1  1 

{  n+1 

1 

l-2-r 

e 

n+1  i 


The  maximum  distance  from  any  point  in  N.  to  X.  is  — — -  -~ 

i  l  n+1  n+1 

thus  the  maximum  distance  from  any  point  in  N.  to  X  is  d, 

2 

d  <  (7TTT  )  +  — ^—2  [pHs^+n+1]  ,  or 

(n+1) 


d  <  p 


1  1 

'  /ml)  ,  n  ,  nr\ 

(n+1)2  ' 

|  *  n+1  /  +  n  +N  n/ 

R 


T’ 


which  is  less  than  p  for  n  >  1.  Thus  N. 
hood  about  X  is  in  R_  . 


since  a  p-neighbor- 


We  will  now  solve  the  following  problems,  for  all 

X  e  X  : 
i  P 

b.  Solution  Procedure  for  Feasible  Solution  to  a 
Generalized  Program 

Find  a  vector  P  (or  a  convex  combination  of  vectors), 

so  that 


PM  +  Iy+  -  iy"  = 


».  =  1 

P'  y^'  yi  ^  0  ’  P  «-  C  .  (5.39) 


This  is  the  phase  I  procedure  of  a  generalized  program.  The  columns 
P  can  now  be  generated  to  ihe  master  program  (a  linear  program), 


min  w 

i  .  y 


n 

V 

i  =1 


+ 

y,  + 


v 


i=i 


y; 


p 


i 


+ 


+ 


Iv 


ly 


X 

i 


tj  1  >k 


,«v  yj-  yt  _ 0  • 


(5.40) 
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From  the  vector  of  dual  variables  it  ,  the  k  iteration  of  the  general- 

k+1 

ized  program  (5.  40)  generates  a  new  column  P  ,  by  means  of  the  sub¬ 
problem, 


min  — k 

_  it 

P  £  C 


(5.41) 


k+1 

This  subproblem  is  equivalent  to  finding  a  vector  function  u  (t)  that 


k'  F(T-t) 

minimizes  it  e  Gu(t) 


subject  to  u(t)  e  U  , 


where 


-k' 

it 


(5.42) 


The  problem  (5.42)  for  polyhedral  U  has  previously  been  shown  to  be 
a  parametric  linear  programming  problem  and  is  solvable  by  the  methods 
introduced  in  Chapter  IV, 

k+1  k+1 

The  generation  of  P  from  u  (t)  is 

k+1  rT  F(T-t)  „  k+1,  .  _ 

P  =  I  e  Gu  (t)  dt  . 

Jo 


The  generalized  program  (5.39)  terminates  when 


k 


w 


< 


_! >-(S-)2 

(n+1)  yn+1 J 


for  some  iteration  k  of  (5,40).  We  know  that  the  minimum  value  of 

w  for  all  P  e  C  is  zero,  since  X.  t  R.  and 

l  T 


P 


i* 


e  C  . 


82 


1 


Therefore 


_i  *  .  + 

P  p  +  ly 


iy  =  x. 


=  1 


M>  yt>  y,  >  0,  has  a  solution  with  all  y*,  y  =  0.  The  terminating 


ii-'  "ii 

condition  for  each  part  of  the  phase  1  procedure  terminates  a  general¬ 


ized  program  at  a  suboptimal  solution  with  the  objective  value  some 
specified  distance  from  its  optimal  value.  Thus  the  generalized  program 
terminates  in  a  finite  number  of  steps. 

Once  the  value  of  w  becomes  low  enough,  the  solution  to 


the  phase  I  procedure  for  each  X  must  be  recovered.  From  the  final 


solution,  for  each  X.  phase,  let 


i* 

u  (t) 


I  ul<t)|'i 


and 


i=l 


V 


1=1 


for  each  i  =  0,  1,  .  .,  n.  Also  note  that 


k 

y 


p  ‘i 


i  =1 


v  rl  F(T-t)  „r  i,  , , 
N  /  e  G[u  (t)] 


dt 


Pi 


i  =1 


-f 


F(T-t)  „  i*  .  , 

e  Gu  (t)  dt 


Proposition  5.7.  Ttie  set  of  vectors  P  and  controls  u  (t)  con¬ 


stitute  a  nondegenerate  icasiblc  solution  to  the  set  of  equations, 
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m « 


1 


a  >  o  ,  Vi  , 


and  by  retransforming  and  rescaling,  p. 


CT  >  0,  Vi  for  (5.43). 

Q.  E.  D. 


c.  Application  to  Control  Problems 

The  above  procedure  may  be  used  to  initiate  the  optimum 
control  problem  when  feasibility  is  known.  However,  in  cases  where  the 
existence  of  a  feasible  solution  is  not  known,  another  phase  I  procedure 
can  be  used.  This  procedure  will  determine  feasibility  and  in  the  pro¬ 
cess  provide  an  initial  feasible  solution. 

For  the  fixed  end  point  problem,  a  solution  with  w  =  0 
to  the  generalized  program, 


min 

b.y 


w 


n  n 


\ 


+ 


+ 


i=l 


i  =1 


P|  +  Iy  +  ly  =  S 


i 


P  C  c  . 


(5.  44) 


implies  that  a  vector  P  t  C  exists  which  provides  a  feasible  solution 

to  the  control  problem.  If  the  optimal  solution  to  (5.44)  has  a  value 
¥ 

of  w  >  0,  a  feasible  solution  does  not  exist  to  the  control  problem. 

Theorem  5,2.  If  at  any  stage  k  in  the  solution  of  (5.44),  the  value 

of  w'  +  c  0,  the  original  control  problem  is  infeasible,  where 
bk  is  the  objective  value  of  the  subproblem  to  the  generalized 
program  (5. 44) . 
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Proof  of  Theorem  5.2. 


th 


Consider  the  master  program  at  the  k  stage,  as;  max  A 


—  + 


subject  to  UqA  +  P1(i1  +  .  . .  +  P  pk  +  iy  "  !y  -  Sv  -  0 


Ml  +  ...  + 


y**  y[>  pi  >  0  • 


=  i 


(5. 45) 


where 


P1  - 


S  = 


■  1 

1 

1" 

■  -1 

-1 

-1 " 

I  = 

,  I  = 

I 

_ 

I 

- 

Thus  A^  =  -w^  and  the  dual  variable  to  (5.45)  is 

_k'  /  k  k'  k  k  \ 

n  -  ^o>  it  ,  rtn+1  *  nn+2  /  ' 

By  using  the  duality  theorem  of  linear  programming  and  the  dual  to  (5.45) 

,  k  k  k 

A  =  ;t  ,  +  it  „ 

n+1  n+2  . 

Also 


k  min  k' 
6  =  P  £  C  ^ 


0 

p 

1 


min  k'  k 

:P  £  c  p  +  Vi 


Since  S  is  always  in  the  basis  of  (5.45) 


n  0=0  implies 


k'  k 

n  (_S)  =  ~nn+2  ' 


(5.46) 


By  hypothesis, 


n  .  tk  k  min  k'  k  k 

0  <  b  +  w  =  _  it  P  +  it  _  +  w  , 

P  £  C  n+1 


n  .  J<  k  k'  k  k 

0  <  0  -  W  <  It  P+JT  „  +  W 

-  n+1 


for  any  P  e  C 


o  •  k  k  k  k 

Since  w  =  -A  =  -it  -  n  „  , 

n+1  n+2 


„  .  c  k  k  k_.  k 

0  <  b  +  w  <  rt  P  -  it  g  >  and  by  (5.46) 


_  k  k  k'  k' 

0  <  o  +  w  <  -  P  -  ;t  S  ,  or 


0  <  Ck  +  wk  <  irk'(P-S)  , 


for  all  P  . 


Thus,  for  any  P  e  C  , 


0  <  n  (P-S)  , 


whicn  implies  that 


^  0  and 


P  -  S  ^  0  for  all  P. 


87 


Thus  there  is  no  admissible  control  function  u(t)  which  can  generate 
a  feasible  solution  to  the  control  problem. 

Q.  E.  D. 

However,  if  at  any  iteration  k,  w  =0,  the  current 
solution  is  a  feasible  solution  to  the  control  problem  and  phase  II 
(the  optimization  phase)  of  the  generalized  program  can  be  initiated. 

Since  this  phase  I  procedure,  unlike  the  previous  one,  is  not  necessar- 
ily  a  finite  process,  the  optimization  phase  may  begin  when  w  <  t,  some 
small  positive  number  and  the  desired  final  point  not  precisely  attained. 
For  any  practical  control  problem,  when  feasibility  is  not  known,  a  point 
at  some  arbitrarily  small  distance  away  from  a  determined  fixed  point 
would  be  an  acceptable  terminal  point  for  the  control  problem.  Thus, 
the  phase  I  procedure  would  be  finite  even  when  feasibility  is  not 
assumed. 

For  the  variable  end  point  problem,  phase  I  procedures 
are  much  simpler.  For  example,  when  the  desired  final  region  &  is 
constrained  to  lie  in  some  r-neighborhood  about  a  determined  point  S, 
the  phase  I  procedures  outlined  previously  are  used  (r  <  p) .  If  fea¬ 
sibility  is  assumed,  a  series  of  reachable  points  in  the  r-neighborhood 
should  be  chosen,  and  a  phase  I  procedure  identical  lo  the  first  one 
discussed  in  this  paper  would  provide  a  nondegenerate  feasible  solution. 

If  a  nondegenerate  solution  is  not  necessary,  then  a  procedure  identical 
to  that  for  finding  the  existence  of  a  feasible  solution  can  be  used. 

In  this  case  S  is  allowed  to  be  the  right-hand  side  of  (5.44)  and  the 
algorithm  is  terminated  when  w  <  r,  as  the  following  theorem  points  out. 

k  * 

Theorem  5.3.  If  w  ;  r,  then  the  solution  P  to  (5.44)  at  stage  k, 
which  produces  the  value  w  ,  is  a  vector  which  lies  in  an  r-neighbor- 
hood  about  S,  and  it  is  a  feasible  solution  for  the  control  problem. 
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Proof  of  Theorem  5.3. 
Since  w  <  r, 


V  y+  + 


1  +  X  y~i  =  V  |(P*  -  S.)|  <  r  , 


+  /  ( -  S^)  <  r,  and 


|P*  -  S|| 


I(Pi-Si)2  <  X 


+J(P1  -Si)2<r  • 


Q.  E.  D. 


A  general  phase  I  procedure  for  the  variable  end  point 
problem  can  be  used  to  determine  feasibility  as  well  as  to  find  a  fea¬ 
sible  solution  when  it  exists.  IVe  consider  the  problem 


\  +  V 

'  y .  +  }  y . 


subject  to 


P,  +Iv  -  Iv  + 


P  t  C 


s  <-  &  >  or  equivalently, 


(5.  '17) 
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Theorem  5.4.  If  at  any  stage  k  of  the  initial  phase  procedure  for  the 

"  .  k  k  k 

variable  end  point  problem  w  +6  +  A  >0,  the  original  control 

problem  is  infeasible. 


Proof  of  Theorem  5,  4. 

Letting  nk  =  («£,  ,  *„+1.  V2 

duality  theorem,  that 


J  and  remembering  from  the 


k  .  k 
w  =  -A 


^  rj  t 

n+1  n+2 


then,  by  hypothesis, 


„  k  k  k 
0  <  b  +  A  +  w 


min  — k'  P  min  — k'  -S 
P  4  C  '  1  +S  4  *  "  0 


l  i  tilting  in  the  above, 


k  k  k  —  k '  — k ' 

0  <  o  +  +  w  <  ••  + 


n+1  n+2 


for  all  P  c  C  and  S  t  A  . 


k  k  k 
+  +  iv  < 


(-S)  +  ::  + 

n+1 


(P-S)  , 


which  implies  that 


Z  *  0 


and 


(P-S)  £  0 


for  all  P  e  C  and  S  e  A 


Therefore,  there  is  no  feasible  solution  to  the  control  problem. 


Q.  E.  D. 


Theorem  5.4  indicates  an  in feasibi 1 i t y  condition,  and  in 
the  next  theorem  (Theorem  5.5)  a  feasibility  condition  is  presented. 

Theorem  5.5.  If  at  any  iteration  k  the  solution  to  (5.48)  provides 
a  vector, 

k 

*  \  k 
s  =  2  s  vi 
J=i 

1  -)f 

and  a  value  wK  <  r  so  that  S  is  an  interior  point  in  A  and  has 

an  i—  neighborhood  surrounding  S  that  is  also  in  ^ ,  the  original  con¬ 

trol  problem  is  feasible  and  the  solution 


+  Y  i  k 

’  =  Z  P  '  1 


i  -1 


f,  being  a  solution  to 
(5. 48)  I 


is  a  feasible,  reachable  point  . 


Proof  of  Theorem  5.5. 

k 

Since  w  <  r, 


the  vector 


a  V 


i  =1 
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has  tlie  property 
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While  discussing  the  optimization  phase  oi  the  generalized 
programming  formulation  of  the  optimal  control  problem,  we  will  con¬ 


sider  only  the  fixed  end  point  problem. 

To  avoid  degeneracy,  it  is  desirable  to  start  the  optimization 
phase  with  a  set  of  n  +  1  vectors  P*  that  provide  a  nonsingular 
basis, 


B 


0 

0 


P1 


1 


P2 


1 


which  is  feasible  for  the  program, 


max  A  ,  -1 

p  V  +  p  4  + 

subject  to 


-n+1 
+  P  , 


n+1 


=  S 


>1  + 


^'n  +  1 


=  1 


■>  0 


where 


P1  = 


r 


f  (  X  ,  u  )  dt 


where  u  ( t )  t  1'  generates  the  vector,  P1 ,  and 


(5.  52) 


S  = 


These  vectors  are  immediately  available  when  using  the  phase  1 
procedure  discussed  initially.  For  other  phase  1  procedures,  once  a 
feasible  solution  is  found  and  assuming  a  neighborhood  about  that  solu¬ 
tion  is  also  feasible,  the  initial  phase  1  procedure  may  be  used  by 
generating  t  lie  right-hand  sides  in  a  similar  manner  about  the  known 
feasible  solut ion. 


1 


Restating  the  fixed  end  point  optimal  control  problem  as 


x  =  Fx  +  Gu 


x(0)  =  0,  x(T)  =  S 


x  £  En  ,  u(t)  e  U  c  ,  tft  , 


where 


U  =  { u|  Au  >  b]  , 


and 


min 
u  ( • ) 


-f 


f(x,u)  d t  , 


where 


f'x  +  g' n 


f(x,u)  =  /  f’x  +  g’u  +  £  |u.| 


i  =1 


fQx  +  gpU  +  u’Qu 


we  can  define  F  and  G,  as  before. 

When  a  set  of  vector  control  functions  u  (t)  is  given  from 
the  phase  I  procedure,  the  vectors  P1  must  be  generated. 


P1 


f  (f 
Jo  ° 


x  +  g'u)  dt  +  I  f(u)  dt 
0 


r 


or 


T  )  -T 

_t  Gu(t)  dt  +  /  f(u)  dt  P 


f  eF(T 

Jq 


■'0 


Ft 

Since  the  matrix  c  must  be  provided  for  the  phase  1  method,  the 
matrix, 
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is  easily  shown  to  be 


L°  i 


by  considering  the  system 


x0(O  =  fQx  x  c  E 


x  (t)  =  Fx 


where  x(t)  =  eF^1  ^  x (  0 ,  and 


x  (t)  =  f qX ( t )  =  f^eF(l"l)  x ( t )  . 


Letting  x 


F(t-i)  . 
c  x  (  t ) 


,,  F(t-i) 
f0° 


F( t -1) 


v° 


From  (5.52)  we  see  that  the  linear  equations  in  ;  always 
insure  that  a  feasible  solution  exists  for  the  control  problem.  The 
simplex  method,  when  applied  K  the  master  problem,  maintains  primal 
feasibility  at  all  times  even  when  augmenting  the  linear  equations  with 

1  _  ,  .  .  ,  .  .  ,  . .  . _  r  .  i.  i  J 


a  vector  P 


Thus  at  any  time  in  the  execution  of  the  opt  imi  /.a  t  i  on 


phase,  a  feasible  solution  is  available  from  the  current  basis.  Also 
as  will  be  shown,  a  bound  on  the  optimal  solution  is  provided  at  each 
stage. 


T»t  ■ 


k  k  k 

In  the  following  let  J  =  -A  ,  where  A  is  the  objective 
value  of  the  master  problem  in  the  iteration. 


Theorem  5.6.  During  any  k  iteration  of  the  optimization  phase  of  the 
generalized  program,  the  optimal  value  of  the  cost  function  J(u  ) 
satisfies  the  following  inequalities  [2]: 


k,~.  k  ,  *  k.^.  k  ,k  , _ 

J  (u)  +  b  <  J(u  )  <  J  (u)  <  J  =  -A  ,  (5.53) 


where 


Proof  of  Theorem  5,6. 

Consider  the  equivalent  linear  program,  as  before, 


max  A 

h 

V  +  P  pl  +  P  1  2  +  ■  '  ,  +  P  pk  ~  Sv  =  0 


"1  +  p2  +  •••  +  pk 


v  =  1 


pi  -°  • 


(5. 54) 


The  solution  to  (5.54)  is  A  and  by  the  dual  theorem, 


,  k  k  k 

A  =  -  +  r 

n+1  n+2 

__  k  *  /k  k*  k  k  \ 

where  n  =  (n  ,  ,t“  ,  ,  2 )  is  tlle  dual  variable  to  (5.54). 

n  =  1,  the  subproblem  has  the  solution, 


ck  min  1  .  k'  k 

b  =  <  J(u)  +  ir  P  +  rr 

u(t)  t  U  J  n+1 


Since 


T 


Thus  for  the  value  u  , 


k  *  k'  *  k 

6  <  J(u  )  +  tt  P  +  «n+1  ,  and 


kkkk  *  k'*k  k  k 

Ak  +  6k  =  /  +  b"  <  J(u  )  +  rr  P  +  Vl  -  nn+1  -  «n+2  . 


k  k  ,  *  k'  *  k 
J  +  5  <  J(u  )  +  it  P  -  nn+2 


Since 


k'  0  k  . 

=  -n  S  +  nn+2  =  0  , 


Jk  +  5k  <  J(u*)  +  nk' (P#  -  S)  . 


u*  is  the  optimal  u,  then  P  is  feasible  and  P  =  S.  Thus, 


Jk  +  bk  <  J(u*>  <  Jk(u) ,  where  Jk(u) 


is  the  current  solution  and  u  is  a  feasible  control,  and  the  right 

* 

hand  inequality  follows  immediately  since  J(u  )  is  the  minimum  cost, 
Since  J(u)  is  convex  in  u, 


.  k ,  *  .  k 

J  tu)  <  J 


Therefore 


Jk ( u)  +  6k  <  Jk  +  tk  <  J(u  )  <  Jk(u)  <  Jk 


(5.55) 


Q.  E.  D. 


k  k  ^  k  ■M' 

Corollary  5.1.  When  6=0,  J  (u)  =  J  =  J(u  ). 

Proof  of  Corollary  5.1. 

The  proof  follows  from  (5.55),  since  equality  holds  throughout. 

Q.  E.D. 

Note  that  the  value,  A  ,  of  the  master  problem  is  an  approximation  to 
the  current  solution  at  iteration  k.  At  any  stage,  the  solution  de¬ 
fined  by 

k 

u(t)  =  ^  |iX(t> 

i=l 

has  a  cost  J(u)  which  by  convexity  is  smaller  than  Jk. 

Although  6  does  not  necessarily  increase  monotonically  to 
0,  it  does  so  for  a  subsequence  of  k.  Thus  the  best  bound  from  pre¬ 
vious  iterations  should  be  retained  until  a  better  bound  is  attained. 

The  current  value  (at  iteration  k  of  the  generalized  program)  of  6 
may  be  used  to  provide  a  stopping  condition  for  termination  of  the  op¬ 
timization  phase.  By  observing  the  value 


we  can  determine  the  maximum  percentage  by  which  the  objective  function 
can  decrease  for  the  optimal  solution,  and  we  are  assured  that  the  cur¬ 
rent  solution  is  feasible  to  the  original  control  problem. 

We  now  present  a  flow  chart  of  the  generalized  programming 
solution  to  the  optimal  control  problem.  We  will  use  the  fixed  end 
point  problem  for  an  example,  assume  a  p-neighborhood  about  S  is  in 
R^,  and  apply  the  long  form  of  the  phase  I  procedure. 
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.  JUL-  -  &£».»■  , .  -  « 


„  ,  ,  „  F(T--t)  ,  F(T-t) 

Calculate  e  and  e 

(as  real  matrices  with  elements  being  functions 

of  t)  or  have  matrices  available 


|  Initiate  phase  I 

_ ill _ . 

S  =  S - — -  1  +  %  e  ,  i  =  0,  1, 

i  n+1  n+1  -  2  i 

....  n  . 

i 

Set  i  =  0  - 

© 

- -  Replace  i  by  i  +  1 

Q  |  Replace  k  by  k  +  1 


FROM  SUB.  PROB. 


lin  w  V  +  \  - 

i.  y  =Zyi+iyi 

Subject  to  P1^ 


k  -f  ■ 

Subject  to  plMi  +  •  •  •  +  p  Mk  +  iy  -  iy 


Ul  +  ••■  +  pk 


Mi-  yt.  yj  >  0 


is  w<  uk)(£iY  ? 


©Calculate  it  ,  dual  variable  to 
lirear  program 

YES  t  TO  START  OF  SUBPROBLEM 

- 1 - 

- jl .?  i  =  n  ? 

|  YES 

GO  TO  OPTIMIZATION  (BLOCK  8) 

GENERALIZED  PROGRAMMING  ALGORITHM  FOR  LINEAR 
OPTIMAL  CONTROL. 
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OPTIMIZATION 


TO  SUBPROBLEM 


1  01 


Sl'BPROBLEM 
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1 


The  inputs  to  the  flow  chart  are, 

System  matrices  F,  G 

Loss  functions,  f(x,u)  =  fg(x,u)  +  f(u) 
where  fQ(x,u)  =  f^x  +  g^u 

Parameter,  p. 

Final  time  T  [assuming  initial  time  0] 

Final  state  S  or  S  - 

L  T  oj 

Polyhedral  matrix,  A 

Right-hand  side  vector,  b 

Unit  vectors,  U  ,  e_ ,  e„,  ....  e 

012  n+1 

eo  =  ° 

the  vector  1^  . 

To  retain  the  continuous-time  aspects  of  the  control  problems, 

this  algorithm  requires  explicit  knowledge  or  availability  of  the  matrix 
Ft 

e  ,  and  its  time  derivatives. 

It  is  well  known  [4]  that  the  components  of  this  (n  X  n) 

matrix  can  be  expressed  as  polynomials  of  an  order  less  than  or  equal  to 

Ft 

n  with  an  exponential  multiplying  factor.  The  knowledge  of  e  is 
required  for  determining  the  functions  /(t),  used  in  the  parametric 
programming  subproblem,  and  for  determining  the  vectors  P,  given  a 
control  function  u(t),  over  an  interval. 

We  can  express  the  fundamental  matrix  as 

m-1 

•"  -  2  v»  'k  ■ 

k=0 

where  m  is  the  degree  of  the  minimal  polynomial  of  F.  Note  that 
m  <  n,  if  F  is  (n  x  n) .  The  algebraic  equations  determining  O' 

k 

are 
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m  a  =  0(t) 


where  Of  is  an  m-vector  and  0  is  a  vector  with  elements  of  the  form 
^  ®  k  t 

t' e  ,  k  =  1,  0;  £  =  0,  . ..,  m  -  1,  where  the  s's  .re  the 

K  K 

eigenvalues  of  F  and  m^  is  the  multiplicity  of  the  eigenvalue,  s^. 

M  can  be  shown  to  be  nonsingular.  Thus  Of  is  composed  of  linear  com- 

K 

binations  of  the  elements  of  0(t),  which  are  themselves  members  of 
the  class  of  solutions  to  homogeneous,  constant  coefficient,  linear  dif¬ 
ferential  equations.  Thus  any  vector 


7(1) 


,  F(T-t) 
rr  *  e 


G  , 


where  it  is  a  real  vector  and  G  is  a  real  matrix,  has  components 
which  are  members  of  the  class  of  solutions  to  the  homogeneous,  constant 
coefficient,  linear  differential  equations. 

As  will  be  proven  in  the  next  chapter,  the  functions  generated 
for  the  linear  loss  case,  minimum  fuel  problem,  and  minimum  time  problem 
are  piecewise  constant  functions.  Also,  for  the  quadratic  loss  in  con¬ 
trol  problem,  the  generated  function  u(t)  is  shown  to  be  expressed  as 
a  linear  combination  of  the  Q^s  for  a  finite  interval  of  t.  Thus  the 
components  of  the  vector  integral 


f  l2  F(T-t)  „  ,  ,  J 
/  e  Gu(t)  dt 


can  be  represented  by  a  sum  of  imegrals  of  the  form, 


dt 


which  when  integrated,  is  equivalent  to 


a 

s 


.  2  st 
t  e 
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(5. 56) 


It  should  be  noted  that  all  components  of  e 


will  be  real,  even  for 


Ft 

complex  eigenvalues,  since  cancellations  of  the  complex  part  occur. 

Ft 

Since  the  values  of  e  and  its  first  n  -  1  derivatives 
are  needed  at  only  a  finite  number  of  points  (undetermined),  these  val¬ 
ues  may  be  provided  by  an  analog  computer.  This  suggests  (but  not  re- 
strictively)  the  use  of  hybrid  computers  for  the  algorithm.  The  analog 
computer  could  be  used  to  supply  values  of  /(t)  at  specified  instants 
and  to  compute  the  vectors,  P[u(t)],  while  a  digital  computer  could 
be  used  to  solve  the  linear  and  quadratic  programming  problems  in  the 
master  and  subproblems. 

To  show  that  the  algorithm  is  computationally  feasible,  we  will 
show  that  each  step  or  block  in  the  flow  chart  is  solvable  by  a  finite 
number  of  iterations.  Although  convergence  of  generalized  programs  may 
be  an  infinite  process,  a  suboptimal  solution  as  close  as  desired  to  the 
optimal  solution  is  achievable  in  a  finite  number  of  iterations  of  the 
subproblem.  However,  when  the  reachable  set  is  a  polyhedron,  the  gen¬ 
eralized  program  converges  in  a  finite  number  of  iterations  of  the  sub¬ 
problem.  The  number  of  these  iterations  is  less  than  or  equal  to  the 
number  of  extreme  points  of  the  polyhedron. 

To  demonstrate  the  finiteness  of  t he  executions  at  each  stage 
in  the  algorithm,  wo  will  show  finiteness  for  each  block  of  the  flow 
chart  for  the  basic  algorithm  (note  that  the  block  numbers  designated 
coincide  with  those  on  the  flow  chart.)  It  is  also  noted  where  an  ana¬ 
log  computer  may  be  substituted  when  hybrid  computations  are  desired. 

Ft 

Block  1  ;  If  che  matrix  e  is  not  available  as  an  input,  its  deter¬ 
mination  may  be  obtained  using  an  eigenvalue  analysis  of  the  matrix  F; 

routines  of  this  nature  are  available.  After  the  eigenvalue  analysis, 

F(T-t ) 

a  set  of  linear  equations  must  lie  solved  to  find  c  in  terms  of 

a  finite  sum  of  multiples  of  F.  The  analog  computer  may  be  utilized 

Ft 

for  determination  of  e 

Block  2  :  The  extreme  points  of  a  simplex,  in  the  reachable  set  surround¬ 
ing  the  desired  point  S,  is  a  computation  involving  addition  and  sub¬ 


traction  of  the  available  vectors,  some  of  which  are  unit  vectors. 


Block  5  :  The  solution  to  the  phase  I  master  problem  at  any  k  for  any 
right-hand  side  is  a  linear  program  that  has  an  initial  basic  solution 
immediately  available,  i.e., 

Mi  =  0,  Vi 

y*  =0,  y^  =  -S.  ,  for  S  <  0 

yi  =  Si’  yi  =  °  ’  for  S,  >  0  . 

The  number  of  rows  in  this  linear  program  is  n  +  1  for  any  stage  k, 
even  though  the  number  of  columns  is  variable  but  always  finite. 

Block  6  :  Additional  columns  are  added  to  the  master  program  of  phase 
I  until  the  value  of  w  is  less  than  a  required  strictly  positive  num¬ 
ber.  Since  the  minimum  value  of  w  is  zero  and  since  w  decreases 
monotonically  and  strictly  decreases  on  a  subsequence  of  iterations,  for 
some  specified  positive  number,  the  value  of  w  will  be  smaller  than 
this  number  after  a  finite  number  of  iterations.  This  is  a  basic  result 
of  generalized  programming  problems  and  shows  only  a  finite  number  of  col 
umns  are  used  for  the  master  program. 

Block  7  :  The  calculation  of  the  dual  variable  of  the  linear  program  in 
the  master  problem  is  a  result  of  the  solution  procedure  for  the  linear 
program  and  requires  little  or  no  additional  computation. 

BlockS  :  A  vector  addition  provides  the  column  vectors  to  be  used  in 
the  master  program  after  the  phase  I  procedure  is  completed. 

Blocks  9  and  13  :  The  determination  of  the  vector  P  is  achieved  by 
intregation.  However,  due  to  the  structure  of  the  integrand,  special¬ 
ized  (finite  and  exact)  integration  schemes  are  possible.  The  integral 
is  broken  into  a  finite  sum  of  definite  integrals  (corresponding  to  a 
finite  number  of  switching  points)  whose  end  points  are  calculated  by 
the  analog  computer  or  by  formula  substitution  as  suggested  by  Eq.  (5.56) 


Block  11  :  The  master  program  for  the  optimization  phase  is  a  linear 
program  with  a  fixed  number  of  rows  (at  most  n  +  3)  and  a  variable 
number  of  columns.  Although  the  number  of  columns  may  be  infinite,  for 
any  practical  problem  and  within  limits  of  the  computer's  accuracy,  no 
more  than  a  finite  number  of  columns  are  generated  before  achieving  a 
solution,  indistinguishable  (within  computer  accuracy)  from  the  optimal 
solution.  The  simplex  method  should  be  used  with  the  starting  solution 
to  each  interation  being  the  final  solution  of  the  previous  iteration. 
The  first  vector  to  be  added  to  the  basis  is  the  vector  generated  from 
the  subproblem  (if  optimality  has  not  already  been  achieved). 

Block  12:  The  solution  of  the  parametric  programming  problem  is  dis¬ 
cussed  in  Chapter  IV.  The  solution  has  a  finite  number  of  executions; 
cycling  is  avoided  due  to  the  lexicographic  ordering  rules  and  normal 
degeneracy  perturbation  techniques  available  for  linear  programming 
codes  and  complementary  pivot  theory  methods. 

All  other  steps  are  either  logical  programming  steps  or  simple 
calculations.  Thus  since  each  step  requires  a  finite  number  of  execu¬ 
tions,  each  iteration  of  the  master  problem  and  its  corresponding  sub¬ 
problem  (of  phases  I  and  II)  require  a  finite  number  of  executions. 

The  solution  of  phase  I  is  finite  since  only  a  finite  number 
of  columns  must  be  generated  for  the  n  +  1  generalized  programs  used 
for  the  solution  to  each  step  of  phase  I.  The  algorithm  may  be  termi¬ 
nated  at  any  stage  in  phase  II  yielding  a  feasible  control  with  a  bound 
on  how  much  its  objeptive  value  can  differ  from  the  optimal  objective 
value. 
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Chapter  VI 
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By  using  the  previously  presented  notation,  the  Hamiltonian  can  be  defined 
as 


H  =  \|r'  Fx  +  iJr'Gu  +  \|f'  f  ( u)  UQ 


(6.1) 


x  =  H—  .  and 
* 


♦  =  ”Hx  > 


„(t) 

*0 

where  \|r(t)  = 

and  \|Kt)  = 

• 

V) 

\|r  (t) 

_  n  J 

(6.2) 

(6.3) 


Pontryagin's  maximum  principle  states  that  (6.2)  and  (6.3)  must  hold 
* 

while  the  optimum  u  (t)  satisfies 


H(*|», 


x, 


* 
u  ) 


sup 

u(t)  e  U 


H( \J/,  x,  u)  , 


(6.4) 


for  a  given  \|r,  x,  or 

H(\)r,  x,  u  )  >  H(\Jr,  x,  u) 


all  u(t)  t  U  . 


(6.5) 


Let  the  optimal  dual  solution  to  the  generalized  program, 


max  A 
P  t  C 

subject  to  UqA  +  Pu  =  S 

P  =  1 

P  £  C  , 


be 


_* ' 


)  ; 


and  let 


t(  O 


F(T-t) 

-it  e 


(6.6) 
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• -  wv, ~  .rvi v ;  7- 1 - » ■ 


#  *  _ 
where  n  =  (tt^,  ...,  jO .  Thus  it  is  obvious  that  \j/(t),  as  defined, 

provides  a  solution  to  (6.3).  To  show  that  this  solution  is  non-trivial, 

it  is  sufficient  to  show  that  \]i  (t)  ^  0*  From  the  results  obtained  in 

up( T_t) 

Chapter  V,  the  first  column  of  e  is  the  unit  vector  U  .  Thus, 

0 


\|r  (t)  =  -tTq  ,  for  all  t  . 


We  also  know,  from  generalized  programming,  that 


n  UQ  =  1  , 


therefore,  rr*  =  1  and  w  ( t)  =  -1,  for  all  t. 

0  T0 
■■■■  )(■ 

Since  it  is  the  dual  solution  to  the  generalized  program,  it 
satisfies 


min  P  P 

-  -  It  <  IT 

P£  c  ,  -  , 


P  6  C  , 


or,  equivalently, 


min  — P(u) 
u(t)  e  U  n 


:(u)  ^  P(u) 

<  n 

1  J "  L  1  . 


Vu(t)  t  U  .  (6.8) 

The  above  inequality  is  equivalent  to  the  subproblem  of  the  generalized 
_* 

program,  when  rr  is  the  current  dual  variable.  Equation  (6.8)  may  be 
restated  as 

*  *  F(T-t )  * 

5T  f(u  )  +  IT  e  K  ’  Gu  (t) 


*  FfT-t )  — 

<  it  f(u)  +  rt  e  '  Gu  ( t) 


for  all  u(t)  e  U,  and 
t  t  [0, T]  , 


I——.  1- - 


or 


*  * 
-it  f  ( u  ) 


it 


F(T-t) 

e 


_  * 

Gu  ( t) 


>  -itQf(u) 


F(T-t) 

e 


Gu(  t) 


By  expressing  the  above  in  terms  of  ^F(t)  and  by  using  Eqs.  (6.6)  and 
(6.7),  the  inequality  is  equivalent  to  (6.5)  or  Pontryagin's  maximum 
principle.  Thus  the  generalized  programming  necessary  conditions  are 
equivalent  to  Pontryagin's  necessary  conditions  for  the  same  problem. 

To  complete  the  analogy  between  Pontryagin's  necessary  conditions 
and  the  generalized  programming  optimality  conditions  for  control  prob¬ 
lems,  we  include  here  a  discussion  of  t^e  transversal! ty  conditions  for 
sets  i  which  are  convex  smooth  manifolds. 

— 

Notice  that  the  vector  rt  is  equivalent  to  the  vector  \|r(T) . 

Also  an  optimality  condition  for  free  (initial)  final  point  problems 
[as  shown  in  Eq.  (5.47)]  is 


_*< 

it 


>  0  , 


for  all  S  e  &  . 


* 

We  also  note  that  some  S  t  i  has  the  property 

*'  *  * 


* 

since  there  must  be  some  vector  S  in  the  basis  of  the  expanded  master 
linear  program. 

These  conditions  represent  a  halfspace  with  the  hyperplane  defining 

_•* 

it  as  being  represented  by  the  vector  it  .  This  hyperplane  is  a  sup- 

*  * '  -x  * 

porting  hyperplane  to  <4  at  some  S  ,  since  it  S  -  ~‘:n+2’  lies 

completely  in  one  halfspace  of  the  hyperplane.  The  hyperplane  is  also 
a  tangent  plane  to  the  manifold  &  (when  S  is  a  manifold).  These 
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conditions  represent  the  fact  that  \|/(t)  =  -it  is  orthogonal  to  the 

* 

tangent  hyperplane  of  il  at  S  .  This  is  precisely  the  transversality 
condition  described  in  Chapter  II. 

To  show  that  a  solution  to  the  generalized  programming  problem  is 
also  a  solution  to  the  continuous-time  optimal  control  problem  (for 
fixed  end  points),  it  is  assumed  that  we  have  a  finite  set  oi  vectors 
P1,  so  that 


=  0  and 


>0,  all  PeC 


^  p\.,  =  s 


1 


'■i  -° 


(6.9) 


has  a  solution, 


Theorem  6.1.  The  solution 


*  \  i  * 

U  (  t  )  =  \  U  (  t  )  1  ;  . 


t  t  [0,T] 


is  an  optimal  solution  for  t  tic  control  problem. 


Proof  of  Theorem  6.1. 


We  know 


F(T-t) 

\|i(t)  =  -it  e 


satisfies  (6.3)  and  \|r  (t)  =  -1  satisfies  \|j  (t)  <  0,  t  e  [0,T] 


*  F(T-t)  —  * 

f(u  )  +  it  e  Gu  (t)  =  f 


V  *  i 

Z  piu 


(t) 


F(T-t)  - 
+  it  e  G 


5 


*  i 

p.u  (t) 


and,  from  convexity, 


f  ( u  )  <  f 


V'  *  i  #  i 

2  p i u  (t)  <  2^if(u ' 


Since  6  =0,  from  generalized  programming  necessary  conditions 


*  FfT-tl  —  *  * 

0  =  f(u  )  +  it  e  Gu  (t)  +  it 

n+1 


<  +  \  Vl 


*'  F(T-t)  -  i,  ,  V  *  * 

it  e  Gu  (t)  +  )  pj  nn+1 


V-^  *  » 

>  Mj  * 


P1 


=  0 


★ 

Since  it 

n+1 


is  constant  and  since 


Gu(t) 


f(u  )  +  rr 


F(T-t) 

e 


_  -)f 

Gu  (t)  <  f(u)  +  Tt 


F(T-t) 

e 


Vu(t)  €  U  , 


or 


* 

-  f(u  ) 


Tt 


F(T-t) 

e 


Gu*(t)  >  -f(u) 


F(T-t) 

it  e 


Gu( t)  , 


u(t)  e  U  , 


t  e  CO,  T]  , 


which  is  equivalent  to  the  maximum  principle.  Thus  Pontryagin’ s  neces¬ 
sary  conditions  for  optimality  are  satisfied  by  the  solution  to  the  gen¬ 
eralized  programming  problem.  Since  the  set  of  equations  (6.9)  has  a 

* 

solution,  the  control  u  (t)  is  a  feasible  control.  It  remains  to  be 
shown  that 


J(u  )  <  J(u)  ,  V U ( t )  €  U 

which  was  shown  to  be  a  result  of  t C  =  0,  in  Theorem  5.6.  Thus  the 
* 

control  u  (t)  is  an  optimal  control  for  the  continuous-time  control 
problem. 

Q.  E.  D. 

We  will  now  show  that  given  r  t  an  optimal  dual  solution,  a  finite 
set  of  vectors  P1  can  be  found  t  o  provide  a  feasible  solution  to  the 
set  of  equations  (6.9).  This  is  done  for  three  cases,  the  quadratic 
loss  in  control  with  positive  definite  matrix  Q,  t  lie  linear  cases 
satisfying  Pontryagin' s  general  position  condition,  and  finally,  the 
linear  cases  not  satisfying  Pontryagin' s  general  position  condition. 

Theorem  6.2.  If  the  quadratic  loss  control  problem  with  positive  def- 
— — —  ^ 

inite  Q  has  a  feasible  solution,  then  u  (t),  which  provides  a 

solution  to 

1 15 


■  - .....  . 


is  an  optimal  control  for  the  control  problem. 


Proof  of  Theorem  6.2. 

* 

Let  u  (t)  be  a  control  satisfying 


* 

then  u  (t)  solves 


min  u(t) '  Qu( t)  +  /  <  t ) ’  u(t) 


u(t)  e  U  ,  t  e  [0, T]  , 


where  }  (t)  is  generated  by  n  .  The  quadratic  program  with  Q  posi¬ 
tive  definite  has  a  unique  solution  at  each  t.  Thus  there  is  no  other 
u(t)  satisfying 


_*< 

TT 


P(u) 

1 


=  0  . 


By  the  feasibility  assumption  and  by  U  being  a  compact  set,  an  optimal 

* 

control  exists  and  must  satisfy  the  necessary  conditions.  Since  u  (t) 
is  the  only  control  satisfying  the  necessary  conditions  for  optimality, 
it  must  be  the  optimal  control. 

Q.E.D. 

Theorem  6,3.  For  feasible  linear  control  problems  (including  minimum 

fuel  and  minimum  time)  where  F,  G,  and  U  satisfy  the  general 

* 

position  condition  of  Pontryagin,  the  control  u  (t)  satisfying 


1 


is  an  optimal  control. 


Proof  of  Theorem  6,3. 

Since  a  feasible  control  exists  and  since  U  is  compact,  an  opti- 
** 

mal  control  u  ( t)  must  exis*-  and  satisfy  the  necessary  conditions  for 
optimality.  From  the  results  of  Pontryagin,  when  the  general  position 
condition  holds, 


min  — 
u  ( t )  e  U  n 


(6.10) 


has  a  unique  solution  [of  u(t)], 
#  * 

which  we  call  u  (t).  Thus  u  (t) 
necessary  conditions,  and 


except  on  a  set  of  measure  zero  > 
is  the  only  function  satisfying  the 


*  ** 
u  (t)  =  u  (t) 


Q.  E.  D. 


When  the  general  position  condition  is  rot  satisfied  by  F,  G, 
and  U,  the  solution  to  (6.10)  is  not  necessarily  unique  over  a  set 
of  positive  measure.  However,  since  (6.10)  must  be  satisfied  (be¬ 
cause  it  is  a  necessary  condition),  only  its  solutions  need  be  investi¬ 
gated  to  produce  the  optimal  control.  This  is  true,  since  the  problem 
is  feasible,  U  is  compact,  and  an  optimal  control  exists.  By  the 
theory  of  generalized  programming,  any  solution  to  (6.10)  which  is  fea¬ 
sible  for  (6.9)  is  an  optimal  solution,  as  shown  by  Theorem  6.1. 


Proposition  6.1.  There  are  a  finite  number  of  distinct  solutions  to 

(6.10). 
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Proof  oi  Proposition  6.1. 

It  has  been  shown  in  Chapter  IV  that  an  upper  bound  exists  on  the 

number  of  possible  switching  points  of  any  solution  to  the  parametric 

_* 

programming  problem  (6.10),  for  any  value  of  n  for  a  finite  interval 
of  t.  These  points  are  fixed  (given  17*),  and  any  solution  to  (6.10) 
remains  constant  between  any  neighboring  pair  of  switching  points.  There 
are  a  finite  number  of  possible  solutions  for  (6.10),  between  such 
switching  points  (due  to  a  finite  number  of  bases).  Thus  there  are  a 
finite  number  of  distinct  solutions  to  (6.10). 

Q.  E.  D. 


Proposition  6. 2.  The  set  of  points  P,  generated  by  (6.10),  are  extreme 
points  of  a  convex  (bounded)  polyhedron  of  all  P  satisfying 


_*< 

n 


(6.11) 


Proof  of  Proposition  6.2. 

By  the  minimization  procedure  of  (6.10),  extreme  points  are  gen¬ 
erated.  Also  by  the  homogeneity  of  (6.11),  any  convex  combination  of 
the  finite  number  of  extreme  points  satisfy  (6.10).  Thus  the  points 
generated  by  (6.10)  are  extreme  points  of  the  convex  polyhedral  set 
containing  the  solutions  to  (6.11). 

Q.  E.  D. 

Theorem  6.4.  For  linear  loss  problems  not  satisfying  the  general  posi¬ 
tion  condition,  a  finite  set  of  vectors  P1  can  be  found  to  pro¬ 
vide  an  optimal  solution  to  the  generalized  programming  problem. 

Proof  of  Theorem  6.4. 

The  optimal  solution  satisfies 

ix 
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1 


hence  we  have  shown  that  a  finite  number  of  vectors  P  form  the  extreme 
points  of  all  vectors  satisfying  the  Eq.  (6.11).  Since  an  optimal  solu¬ 
tion  exists,  it  must  satisfy  (6.10),  and  therefore,  it  must  be  a  convex- 
combination  of  all  extreme  points  to  the  set  of  vectors  satisfying  (6.11). 
It  can  be  shown  that  an  optimal  control  vector  P*  must  be  a  combination 
of  at  most  n  +  1  extreme  vectors  P1  satisfying  (6.10).  Thus,  the 
finite  set  of  extreme  vectors  satisfying  (6.10)  can  be  generated  and  must 
include,  m  its  convex  hull,  a  feasible  solution  to  (6.9).  Therefore, 
the  optimal  solution  is  determined  by  a  finite  set  of  vectors  p\ 

Q.  E.  D. 

% 

Let  p  be  the  solution  to 

X  *— 1  ~ 

V  +  ^P  +  •••  +  hkP  =  s 


k 


i=l 


(6. 12) 


where  the  P  are  the  extreme  vectors  of  (6.10)  or  (6.11).  Such  a 
solution  exists  from  Theorem  6.4. 


Theorem  6.5. 


* 

u  (t) 


1  <l) 

i  =1 


where  u  (t)  generates  P* , 
control  problem. 


is  an  optimal  solution  to  the 


Proof  of  Theorem  6.5. 

u  (t)  provides  a  feasible  solution  by  virtue  of  (6.12)  and  satis¬ 
fies  the  necessary  conditions  by  construction;  by  Theorem  6.1,  the  cost 
function  is  minimal  over  all  feasible  controls. 

Q.  E.  D. 
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B. 


Characteristics  of  Linear  Loss  Optimal  Controls 


Without  assuming  a  general  position  condition,  we  will  show  that  an 
optimal  solution  to  the  control  variable  for  line  r  loss  functionals, 
including  minimal  fuel  and  minimum  time  problems  is  a  piecewise  con¬ 
stant  function  with  a  finite  number  of  points  o  discontinuity  for  any 
finite  interval  of  time.  Thus,  since  we  are  considering  finite  horizon 
problems  (T  <  °°) ,  an  optimal  control  is  a  piecewise  constant  vector 
function  with  a  finite  number  of  switching  points. 

The  solution  to  the  parametric  linear  program  is  observed  as  being 
a  vector  control  function  that  is  piecewise  constant  and  has  a  finite 
number  of  switching  points.  Thus  any  vector  ,  generated  by  the 
subproblem,  has  the  same  property  for  its  generating  control  function. 
Since  generalized  programming  problems  are  linear  programs  in  the  master 
problem,  and  since  the  number  of  rows  in  the  linear  program  is  less  than 
or  equal  to  (n  +  3) ,  the  number  of  columns  P1  in  any  solution  is  at 
most  (n  +  1)  for  phases  I  and  II  of  the  algorithm. 

Proposition  6.3.  The  columns  P1 ,  generated  for  the  optimization  phase 
of  t he  algorithm  by  the  phase  I  procedure,  are  generated  by  control 
vectors  that  are  piecewise  constant  with  a  finite  number  of  dis¬ 
con  t inui t ies. 

Proof  of  Proposition  6.3. 

In  general,  the  n  +  1  columns  P1 ,  for  i  =  0,  1,  ...,  n  gen¬ 
erated  for  an  initial  feasible  solution  to  the  control  problem,  are  gen¬ 
erated  from  the  ptiase  I  algorithm  for  n  +  1  right-hand  sides.  Titus 
each  P1  is  generated  by  a  set  of  at  most  n  +  1  vectors  P,  and 

each  in  turn  is  generated  ttv  a  piecewise  constant  control  function  witli 

a  finite  number  of  switchings.  If  t  lie  maximum  number  of  switchings  for 
any  control  function  generated  by  the  subproblem  is  M  (M  <  °o)  ,  each 
P1  has  a  control  with  at  most  (n  +  1)M  switchings,  since  these 

controls  are  generated  by  summing  n  +  1  control  functions  with  at 

most  M  switchings  each. 
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Each  new  column  (after  feasibility  is  attained)  is  generated  by  a 
piecewise  constant  control  with  at  most  M  switchings.  Thus  each 
column  P1,  in  the  master  problem  of  phase  II,  has  at  most  (n  +  1)M 
switchings  in  control. 

Q.E.  D. 

Proposition  6.4.  At  any  stage  in  the  iterative  process  of  the  generalized 
program,  the  current  control  solution  is  a  piecewise  constant  func¬ 
tion  with  a  finite  number  of  discontinuities  and  has  an  objective 
value  within  the  bound  of  the  optimal  objective  value,  given  in 
Eq.  (5.53),  of  ok . 

Proof  of  Proposition  6.4. 

The  solution  to  the  master  problem  contains  a  nonnegative  combina¬ 
tion  of  at  most  n  +  1  columns  P1,  each  generated  by  a  control  func¬ 
tion  tl'.at  is  piecewise  constant,  and  each  has  at  most  (n+l)M  switch¬ 
ing  points  (Proposition  6.3).  Thus,  the  combination  of  the  controls  to 
generate  the  solution  has  at  most  (n  +  1 )  2M  discontinuities.  The 
bound  (c  )  was  shown  in  Chapter  V. 

Q.  E.  D. 

Theorem  6.6.  The  optimal  control  generated  by  the  generalized  program¬ 
ming  solution  of  the  continuous-time  problem  for  the  linear  loss 

i 

functionals  (minimum  fuel  and  minimal  time  problems  included  as 
special  cases)  is  a  piecewise  constant  function,  and  it  has  a 
finite  number  of  discontinuities. 

Proof  of  Theorem  6.6. 

If  t lie  generalized  program  terminates  with  a  value  of  =  0  for 

some  stage  k,  then  by  Proposition  6.4,  the  theorem  is  true. 

In  any  case,  given  the  optimal  dual  variables  to  the  generalized 
program  rr  ,  the  optimal  solution  is  a  combination  of  at  most  n  +  1 
vectors  P1 ,  generated  as  solutions  to  (6.10).  The  generating  con¬ 
trols  of  these  vectors  have  at  most  M  switchings,  and  their  combina¬ 
tions  has  most  (n  +1 )M  switchings. 

Q.  E.  D. 
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k  — k 

It  should  be  noted  that  when  6  =0,  the  current  value  of  it  19 

optimal,  and  it  can  be  used  to  determine  an  optimal  control  function  in¬ 
dependent  of  the  current  solution  (but  not  necessarily  distinct). 

Theorem  6.7.  If  the  solution  to  the  parametric  linear  programming  prob- 
•  -  _* 

lem  when  using  it  to  generate  /(t)  is  unique  except  on  a  set  of 
measure  zero,  the  optimal  control  function,  when  non-zero,  is  at  an 
extreme  point  of  the  admissible  control  region. 

Proof  of  Theorem  6.7. 

The  solution  to  a  linear  program  always  occurs  at  an  extreme  point 
of  the  constraint  set.  When  a  change  of  variables  is  made  to  produce  an 
equivalent  problem  for  minimum  fuel  problems,  a  control  of  level  zero  is 
considered  to  be  at  an  extreme  point  of  the  new  constraint  set.  Thus, 
the  optimal  control  is  at  an  extreme  point  of  the  admissible  control 
region  (or  an  equivalent  constraint  set  for  minimum  fuel  or  minimum  time 
problems) . 

Q.  E.D. 

The  previous  theorem  also  implies  that  the  standard  minimal  time 
problem,  and  certain  linear  loss  problems,  have  bang-bang  solutions.  It 
also  implies  that  the  minimal  fuel  solution  is  a  bang-coast -bang  solu¬ 
tion  in  some  cases. 

Pontryagin  C 1 1  has  shown  that  his  general  position  condition  is  a 
sufficient  condition  to  insure  that  the  parametric  linear  programming 
problem  lias  a  unique  solution  almost  everywhere. 

Proposition  6.5.  The  upper  bound  on  the  number  of  switchings  for  the 
linear  loss  functionals  when  the  matrix  F  has  real  eigenvalues, 
the  state  of  t lie  system  is  n,  and  the  matrix  A  of  the  admissible 
control  region  is  m  x  p,  is 

2 

(n+1)  (p-m) 
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Proof  of  Proposition  6.5. 


There  are  at  most  bases  for  the  parametric  linear  program. 

Each  has  (p  -  m)  nonbasic  variables  with  relative  cost  factors  7  (t) 
having  at  most  (n  +  1)  points  at  which  it  becomes  value  zero.  Thus 
if  a  basis  can  be  repeated,  it  can  do  so  no  more  than  (n  +1)  (p  -  m) 
times,  after  which  it  remains  optimal.  Thus  each  column  P1  of  the 
master  program  is  generated  by  a  control  with  at  most 

(n  +  1) (p  -  m) 


switching  points.  Since  at  most  (n  +  1)  control  functions  are  com¬ 
bined,  the  maximum  number  of  switchings  is 


i 


(n  + 


(p  -  m) 


Q.E.D. 


C.  Characteristics  of  Quadratic  Loss  Optimal  Controls 

Since  the  parametric  quadratic  program  has  a  time  (parameter)  de¬ 
pendent  solution  for  t he  control  vector,  the  only  characterization  of 
the  optimal  control  generated  is  in  the  class  of  time  functions  possible 
for  the  solution. 

For  tlie  quadratic  programming  problem,  stated  as 


w  =  Mz  4  q  (  t  )  ,  w ,  z  >  0 ,  w  z  =0  , 

-  i  i 


the  solution  lias  the  form 


where 


r*i 

z 

=  q(t)  +  M 

_  Z  ^ 

w 

=  q( t) ,  and  z,  w  =  0 


123 
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q^(t)  is  merely  a  linear  combination  of  the  original  components  of  q(t) 
which  are  again  linear  combinations  of  the  components 

„  s,  t 
.1  k 
t  e 


for  all  s  eigenvalues  of  F,  and  l  loss  than  the  multiplicity 

K 

of  the  eigenvalue  s  .  Thus  the  solution  to  the  parametric  quadratic 

K 

program  is  composed  of  linear  combinations  of  the  same  elements. 

As  in  the  linear  case,  at  most  (n  +  1)  columns  of  the  generalized 
programming  master  problem  are  used  at  any  stage;  therefore,  the  current 
solution  is  of  the  same  form,  i.e.,  linear  combinations  of  the  elements 

£  Skt 
t  e 


Whenever  a  basis  switch  is  made,  the  linear  multipliers  change  in  the 
linear  combination,  but  the  solution  has  the  same  characteristic  form. 

Proposition  6,6.  At  any  stage  in  the  quadratic  control  problem,  includ¬ 
ing  the  optimal  solution,  the  form  of  each  component  of  the  control 
function  is  a  linear  combination  of  n  +  1  terms  of  the  form 

.  s  t 
A  k 
t  e 


with  only  the  constant  terms  changing  at  each  of  the  finite  number 
of  basis  switches. 


Proof  of  Proposition  6. '6. 

Each  column  generated  by  the  subproblem  has  a  control  function  of 
the  required  form  with  the  finite  number  of  basis  switches,  since  the 
control  function  is  generated  from  the  parametric  quadratic  program. 
Since  at  most  (n  +  1)  columns  are  combined  for  each  solution,  the 
current  control  solution  has  the  same  form.  At  the  optimum,  the  quad¬ 
ratic  program  has  a  unique  solution  for  positive  definite  Q,  and  only 
one  column  is  generated  with  the  generating  control,  which  is  optimal, 
having  the  required  form.  For  positive  semidefinite  Q,  the  solution 
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to  the  quadratic  program  is  not  necessarily  unique,  and  a  combination 
of  at  most  (n  +  1)  control  functions  may  be  required  as  in  the  linear 
case. 

Q.E.D. 

Thi3  algorithm  provides  an  open  loop  solution  to  the  optimal  control 
problem.  Also,  it  should  be  noted  that  no  assertion  is  made  regarding 
the  uniqueness  of  the  solution  in  the  form  of  the  optimal  control 
function. 

Since  Chapter  VII  provides  an  illustrative  example  with  computational 
experience  in  the  linear  case,  we  will  now  present  an  example  showing  the 
form  of  th2  solution  for  the  case  of  quadratic  loss  in  control  [14]. 

Consider  the  system 


» 

x 


i 


=  u 


Thus, 


0  ' 

> 

1  . 

xl(0)  =  0,  x2(0)  =  0 


x  (T) 


and  x2(T)  =  s2  . 


min 

u(0 


dt 


Ml1. 

where  Q  =  —  I  and  is  positive  definite.  If  the  optimum  dual  variable 
to  the  generalized  programming  formulation  of  this  problem  is 


7t 


(“1, 
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the  optimal  solution  can  be  obtained  as  follows: 


find  the  control  u  (t)  so  that 


min 


u(t) |  <  1 


_#> 

7t 


P 

1 


is  achieved  by  u  (t) 


Therefore,  u  (t)  must  solve 


min 

u( 


n  (l  2  *  * 

.)  jj u  +  <”r  V 


)  e 


F(T-t) 


Gu(t)  }  +  r.. 


u(t)  |  <  1,  t  e  [0,  T]  , 


or 


min 
u(- ) 


\  u2  +  jVx ( t) ,  t2<t)j  Gu(t)| 


u(t) |  <  1  , 


where  \|r^(t),  and  ^(t)  are  the  optimal  adjoint  variables  for  all  t. 
Hence  the  minimization  is 


min 

u( 


o  |iu2  +  Vt}'  u| 


\ 


u(t) I  <  1. . 


For  this  problem,  the  solution  is  easily  seen  as 


* 

u  (t)  = 


-sat  \|r2(t) 


-sgn  t2(t)  >  |  'Mt)  I  >  1 
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Note  that 


♦*(«> 


has  the  form 


i.  e.  , 


\}r*(t)  =  at  +  P  . 


Thus  the  optimal  solution  has  the  form  u(t)  =  at  +  P,  for  any  interval 

where  CC  a:.d  P  are  allowed  to  change  at  certain  switching  points. 

Free  Final  State  Problem.  In  conclusion  we  will  consider  the 

problem  where  the  initial  state  is  zero  and  the  final  state  (at  fixed 

time,  T)  is  completely  free.  Since  the  first  through  n  '  rows  of  the 

generalized  programming  master  problem  have  free  right-hand  sides,  the 

slack  variables  for  these  rows  are  always  permitted  to  be  non-zero. 

—* 

Thus,  the  optimal  it  must  be 

-  (l'  °. . Vi>  • 

The  optimal  control  is  then  determined  by  the  subproblem, 


min 

u(.) 


f [x(u) , 


u] 


subject  to  u(t)  e  U  t  e  [0,T]  , 


Thus  the  problem  is  solved  by  one  iteration  of  a  parametric  programming 
problem. 
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Chapter  VII 


Therefore, 
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The  first  master  problem  is  represented  by  the  tableau 

2  2 


min 

M.y 


1=1 


i=l 


0 

0 

1 


0 

1 

0 


■’1 

-1 

0 

0 


J  2 
0 
-1 
0 


-1 

0 

1 


V  V  yi  >  0  • 


The  first  two  rows  correspond  to  states  in  the  dynamic  system,  and  the 
final  row  represents  the  possibilities  of  convex  combinations  of  the 
columns  P*,  generated  by  the  subproblem.  The  optimal  basis  for  this 
linear  program  is 


with 


The  solution  is 

M0  =  yj  =  1.  yl  =  V+2  =  y2  =  ° 

with 
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The  dual  variable  is 

fi°'=  7bb_1  =  (0,  1,  1)  B"1  =  (-1,  1,  0)  , 

t*'  =  («lt  ir2,  «3)]  . 

The  subproblem  for  the  first  interation  is 

min 
P  e  C 

(The  minus  sign  results  from  the  manner  in  which  it  is  generated  from 
7g. )  The  subproblem  is  then  expanded  to 


min 
u(t)  e 


u(t)  dt  -  it 


or 


min 

u(-) 


[-^(3  -  t)  -itg]  u(t), 


|  u(t)  |  <  1  ,  t  6  [0,3]  , 

for  it  =  -1,  it  =  1  . 
1 


The  minimum  is  achieved  by  the  function 

uX(t)  =  -1  t  e  [0,2] 


uV)  =  1 


t  e  (2,3] 


I 

I 

I 

I 

I 

I 

I 

I 

I 

I 

I 

! 


which  generates  a  vector  P  , 


P 


1 


3-t 

1 


u(t)  dt 


-3.5 

-1.0 


The  new  master  problem  tableau  has  an  additional  column  correspond 
ing  to  P1,  i.  e. , 


min  r'  + 

M,yw  =  Iyi+ 

1  1 


^0 


yi  y2 


0  -3.5  1  0  -1  0 

0  -1,0  0  1  0  -1 

1  1  0  0  0  0 


-1 

0 

1 


V  yi’  yi  -  0 


The  optimal  basis  for  this  linear  program  is 


B1  = 


-3.  5 
-1 
1 


0 

0 

1 


and  the  solution  is 


M0 


=  5/7,  =  2/7,  =  2/7,  all  others  equal  zero, 


vi  =  2/7  . 
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To  demonstrate  the  above  fact,  let 


f  0  1  2 

u  (t)  =  |iQu  (t)  +  (t)  +  p2u  (t)  ,  or 


u  (t)  = 


t  =  [0,2] 


t  =  (2,3] 


and  let 


Pf  =  P[uf(t)] 


3  f  3-t 


r*  ° 
Jq  1 


u  (t)  dt  = 


'  -1  ' 
0 


Thus  u  (t)  is  a  feasible  control,  i,e.,  an  admissible  control  bringing 
the  system  from  its  initial  state  to  the  desired  final  state.  Note  that 


J[uA(t)] 


■/’  I*’ 

J0 


(t) I  dt  =  4/3  . 


Since  a  feasible  solution  is  available  consisting  of  a  positive 
combination  of  (n  +  1)  vectors  P,  the  optimization  phase  may  be 
initiated , 


Jtu^t)] 


therefore 


p°  =0,  pj  =  3,  and  P*  =  3  . 


The  initial  master  problem  is  (k  =  2) , 


~.L  -•-■Hi- 


i  ■muwpwywwwr 


or  in  tableau  form, 


max 


max  A 
P 


u0A  ♦  £  -  5 


i=0 


“i  =1 


>  0  > 


1 

0 

0 

0 


0 

0 

0 

1 


3 

-3.5 

-1 

1 


2 

3 

4.5 

3 

1 


0 

-1 

0 

1 


p.  >  0 

l  - 


The  solution  to  this  linear  program  is 


pQ  =  1/3,  =  1/2,  ^  =  1/6  , 


with 


=  2  . 


If  we  define  u  as  the  control 


k 

k  i/«.x 
l^u  (t)  , 

i=0 
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k  th 

where  ji ,  is  the  solution  to  the  k  master  program,  the  inequalities 


in  Theorem  5 .6  become 


■X*  —Jr  ]r  1/ 

J(u  )  <  J(u  )  <  -A  =  j  . 


For  k  =  2  the  solution  is 


J(uk)  =  4/3  ,  -Ak  =  2 


The  dual  variable  jt  is 


7T2'  =  (1,  2,  -4,  0)  , 


and  the  subproblem  is 


,  .k  k+1  k+1 

find  b  ,  u  ,  P  ,  so  that 


.k  -k '  P 

b  =  min  it 


P  £  C 


min  f  '1  k  .  .  /  k  k\  f 3 

u(t>  6  u  l  ”0  lu(t)l  +  [V  ”2j  1  11 


(t)  dt  +  it. 


This  is  equivalent  to  finding  the  solution  to 


|u(t)|  +  J^nk(3-t)  +  nkJ  u(t) 


| u( t) |  <  1 


t  t  [0,3] 


For  k  =  ?,  the  solution  is 
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!-l  t  £  [0,  1/2] 

0  t  £  (1/2,  3/2] 


t  £  (3/2,  3] 


This  solution  produces  a  new  vector 


P3  =  -0.25 


ck  -k ' 

o  =  Jt 


=  -2.5 


Thus,  by  using  the  inequalities  in  Theorem  5.6,  we  arrive  at 


„  — k.  ck  _  — k.  Tk 

J(u  )  +  6  <  J(u  )  <  J(u  )  <  J  ; 


for  k  =  2,  the  solution  is 


-1  .17  <  J(u  )  <  1 .33 


After  each  column  is  generated,  the  master  problem  is  augmented  (all  P 
are  retained) . 

We  will  now  present  the  results  from  a  computer  run  solving  the 

above  problem.  The  program  converged  in  40  iterations,  using  16  place  ac- 

curacy,  on  an  IBM  360/67.  Figure  1  illustrates  the  control  function  u  (t) 

at  iterations  corresponding  to  k  =  2,  3,  4,  5,  10,  15,  30,  40.  Its  cost 

,1(11^)  is  shown  at  each  iteration  in  Fig.  2.  The  optimal  value  J(u  ) 

at  each  iteration  is  shown  in  Fig.  3,  and  the  distance  between  the  two 

k  k 

curves  represents  the  magnitude  of  o  .  The  con\  e>'gen.~e  of  n  and  5 


i 


is  shown  in  Table  1.  In  this  problem,  &  converged  monotonically  to 
zero.  The  convergence  of  rt,  on  the  other  hand,  is  not  monotonic  by 
component  or  component-wise  norm.  However,  it  does  converge  on  a  sub¬ 
sequence  to  its  optimum  value  and  seems  to  monotonically  converge  in  the 
„  i— k 

norm  of  |  n  -  it  |  , 

A  common  penomenon  in  these  problems  was  observed  from  the  gener¬ 
ated  columns  and  their  corresponding  control  functions.  After  the  ini¬ 
tial  iterations,  the  new  columns  seemed  to  be  approaching  a  limit  and 
were  very  nearly  equal  component  wise.  This  is  due  to  the  uniqueness  of 
the  solution  to  the  subproblem  at  (or  near)  the  optimum  dual  variables. 
(Note  that  the  subproblem  has  a  unique  solution  for  every  stage  of  this 
problem.)  Thus  the  control  functions  are  converging  (as  seen  by  Fig.  1) 
to  their  optimum  value,  and  the  state  generatea  by  these  controls  is 
converging  to  its  optimum  desired  value. 

This  similarity  in  the  generated  columns  produces  an  unusual  prob¬ 
lem  in  the  master  program.  The  master  program  develops  into  a  linear 
program  with  approximately  equal  columns  being  basic  or  "nearly"  basic 
columns.  Thus  the  basis  matrix  is  getting  closer  to  a  singular  matrix. 
For  computational  purposes,  this  activity  is  not  very  critical,  since 
it  only  occurs  when  optimality  is  close  at  hand,  a. id  termination  occurs 
before  the  basis  matrix  becomes  singular. 

The  final  solution  computed  for  the  example  consists  of  a  control 

1.000  t  <.  [0,  0.38?  96564] 

0.1176  t  t  (0.38196564,  0.38196754] 

0.0784  t  c  (0.38196754,  0.38196945] 

0.0  t  t  (0,38196945,  2.61802864] 

0.0392  t  t  (2.61802864,  2.61803246] 

0.1176  t  t  (2.61803246,  2.61803436] 

1  .00  t  t  (2  .61803435,  3  .0] 

* 

with  a  cost  J(u  )  =  0.7639320.  If  accuracy  to  within  five  places  is 
sufficient,  then 
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* 

u  (t) 


t  t  [0,  0.38197] 

t  t  (0.38197,  2.61803] 

t  t  (2.61803,  3.0] 


with  J  =  0 .7639  . 

To  compare  this  method  with  other  solution  procedures  for  the  mini 


mum  fuel  problem,  we  observed  the  final  control  function  produces  a  fea 
sible  control  which,  in  turn,  produces  an  objective  value  accurate  to 
15  digits  (double  precision  accuracy)  . 


Although  linear  programming  can  be  used  in  the  discrete  version  of 
the  continuous  problem,  to  achieve  a  solution  as  good  as  that  obtained 
by  the  generalized  programming  method,  the  time  interval  would  have  to 
be  broken  into  more  than  one  million  increments;  these  increments  can 
produce  a  linear  program  with  over  a  million  variables  and  over  a  mil¬ 
lion  rows.  Naturally,  linear  programs  of  that  size  are  too  large  for 
existing  computers. 

By  using  the  generalized  programming  method,  only  linear  programs 
with  rows  approximately  equal  to  the  dimension  of  the  state  space  need 
to  be  solved . 

The  concluding  example  illustrates  the  determination  of  the  exis¬ 
tence  of  a  feasible  solution  to  the  control  problem  to  solve  a  minimum 
time  problem. 


J  dt  with  |u(t)|  <  1  ,  Vt  . 


The  solution  to  this  problem  is  known  to  be  T  =  2 .0  [15]  .  The  solu¬ 
tion  procedure  used  is  to  choose  some  very  large  T  and  solve  the  gen' 
eralized  program 


min 

w 

n.y 


Pp  +  Iy*  -  Iyi  =  S 


M 


=  l  ,  where 


144 


P  is  defined  as  previously.  If  the  problem  is  feasible,  reduce  T;  if 
not,  increase  T  and  continue. 

The  following  table  illustrates  the  number  of  iterations  required 

to  determine  a  feasible  solution,  if  it  exists,  or  the  infeasibility  of 

the  original  problem,  for  any  T.  Note  that  from  Theorem  5.2  when 
k  k 

w  +  5  >  0,  no  feasible  solution  exists  for  the  current  T. 


w  b 

0  0 

0  0 

0  0 

0.05  -0.025 

0.75  -0.25 

Thus,  for  times  when  T  <  2.0,  a  feasible  solution  can  be  found  after 
two  iterations;  and,  for  times  when  T  <  2.0,  the  determination  of  an 
infeasible  problem  can  also  be  discovered  after  two  iterations. 
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